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PREFACE 


This  document  is  a  compilation  of  papers  presented  in  a  symposium  entitled  "Military 
Occupational  Analysis;  Issues  and  Advances  in  Research  and  Application"  at  the  Eighth 
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The  symposium  was  chaired  by  Dr.  Hendrick  W.  Ruck,  then  Chief  Scientist  of  the  Armstrong 
Laboratory's  Human  Resources  Directorate. 

This  document  has  been  edited  by  Winston  Bennett,  Jr.  and  Jimmy  L.  Mitchell.  Final  copy 
editing  was  accomplished  by  Ms.  Nancy  J.  Allin  of  the  Institute  for  Job  and  Occupational 
Analysis. 
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SUMMARY 


Historically,  some  of  the  most  significant  research  and  development  (R&D)  in  occupational 
analysis  (OA)  has  been  accomplished  by  military  organizations  for  military  applications.  Several 
of  the  most  important  OA  methodologies  were  developed  under  military  sponsorship.  Examples 
include  functional  job  analysis,  worker-oriented  approaches  (the  Position  Analysis  Questionnaire, 
the  General  Work  Inventory,  etc.),  and  the  Task  Inventory/Comprehensive  Occupational  Data 
Analysis  Programs  (TI/CODAP)  approach.  Significant  advances  are  continuing  to  be  made  in 
occupational  analysis  for  military  applications.  Many  military  organizations  are  major  users  of 
these  technologies.  The  major  methods  developed  for  military  applications  are  used  in  the 
civilian  world  as  well,  in  bodi  the  public  and  private  sectors. 

This  report  highlights  recent  research  advances  and  applications  of  military  OA  technology.  It 
is  designed  to  bring  military  psychologists  up  to  date  concerning  OA  technology  for  military 
applications  and  the  impact  that  military  occupational  analysis  is  having  on  civilian  practice. 
First,  the  history  of  occupational  analysis  R&D  and  application  in  military  organizations  is 
reviewed,  to  establish  the  context  for  ensuing  chapters  covering  current  military  OA  R&D  results 
and  applications.  The  various  chapters  address  OA  in  both  U.  S.  and  allied  nations’  service 
organizations.  In  addition,  civilian  applications  of  military  OA  technology  are  discussed. 
Finally,  future  directions  for  OA  are  highlighted.  The  potential  use  of  OA  methods  for 
modeling  organizational  structure,  and  for  informing  organizational  interventions  (e.g.,  job 
enlargement,  training,  technostructural  change),  is  also  discussed.  The  final  chapter  includes 
selected  comments  from  symposium  discussants  regarding  a  proposed  research  agenda  for  future 
military  occupational  analysis  R&D. 
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L  MILITARY  OCCUPATIONAL  ANALYSIS:  AN  HISTORICAL  OVERVIEW 


Jimmy  L.  Mitchell 
Walter  E.  Driskill 


Primoff  and  Fine  (1988)  have  observed  that  "the  history  of  job  analysis  is  essentially  a  history 
of  individuals  undertaking  to  satisfy  certain  needs  for  information  about  jobs  in  order  to  deal 
with  practical  personnel  problems"  (p.  14).  They  attribute  the  Civil  Service  reform  movement 
after  the  Civil  War  with  creating  the  drive  for  more  realistic  job  information  and  better 
personnel  selection,  as  an  alternative  to  the  political  spoils  system  for  filling  Federal  jobs.  They 
also  note  the  following: 

Initially,  the  methods  used  to  obtain  the  information  appear  to  have  been  readily 
available  observation,  interviewing  techniques  with  knowledgeable  people,  or  simply 
deduction  based  on  assumed  knowledge.  Only  in  the  past  century  with  the 
introduction  of  scientific  management,  has  there  been  a  focus  on  the  methodologies 
of  fact  gathering  as  such.  (p.  14) 

During  World  War  I,  Walter  Dill  Scott  and  Walter  VanDyke  Bingham  headed  a  Committee  on 
Classification,  a  group  of  noted  psychologists  who  were  charged  with  improving  personnel 
testing  and  placement  for  the  U.  S.  Army.  This  project  resulted  in  the  Army  Alpha  and  Beta 
Tests  for  selection  and  classification  of  recruits  (DuBois,  1970).  Scott  recommended  an 
integrated  personnel  system  with  complete  cross-referencing  between  military  and  civilian  skills 
and  specifications  for  each  job;  he  also  suggested  trade  tests  to  determine  individual  skill  levels. 
Scott’s  goal  was  to  match  people  and  jobs  properly  to  optimize  performance  and  provide  for 
individual  development  (Mitchell,  1988). 

Shortly  after  the  end  of  World  War  I,  Scott  set  up  his  own  company  and  introduced  to  business 
and  industry  the  job  analysis  procedures  developed  in  the  Army.  In  1922,  the  U.  S.  Civil 
Service  Commission  tasked  R.  C.  Clothier,  a  vice  president  of  the  Scott  Company,  to  analyze 
the  jobs  of  1,200  civilian  employees  in  237  occupations  at  McCook  Field,  Dayton,  Ohio,  in 
terms  of  their  duties,  requirements,  and  avenues  of  advancement  (Primoff  &  Fine,  1988).  A 
prime  objective  of  that  study  was  to  classify  occupations  and  identify  grade  levels  within  each 
occupation  (career  progression  paths).  In  the  early  1930’s,  the  Social  Science  Research  Council 
and  the  National  Research  Council  were  concerned  with  improving  job  analysis  for  classification 
and  placement  to  help  remediate  unemployment.  This  emphasis  led  to  the  development,  in  1934, 
of  the  Occupational  Analysis  Program  (OAP;  later  titled  the  Occupational  Analysis  Section  or 
OAS)  of  the  United  States  Employment  Service  (USES).  The  OAS  was  headed  by  C.  L.  Shartle 
until  the  early  days  of  World  War  II  (Primoff  &  Fine,  1988).  A  number  of  psychologists  who 
would  later  pioneer  job  analysis  methodologies  of  their  own  got  their  start  in  the  USES 
occupational  analysis  program,  including  E.  J.  McCormick,  E.  S.  Primoff,  and  S.  A.  Fine. 
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World  War  n  and  Subsequent  Developments 

During  WW II,  psychologists  were  used  extensively  to  help  improve  the  selection  and  placement 
of  military  personnel.  Of  this  effort,  McCormick  (1967)  noted  that  it  "was  then  the  most 
extensive  program  in  personnel  classification  and  assignment  the  world  has  ever  known"  (p. 
121).  He  went  on  to  say  that  "despite  the  criticisms  to  the  contrary,  both  well-founded  and  not, 
it  is  at  least  my  own  personal  conviction  that  those  classification  programs  contributed 
significantly  to  the  effectiveness  with  which  the  military  services  girded  themselves  for  that 
unhappy  period"  (p.  121).  He  cited  the  development  of  a  pilot  selection  program  by  John 
Flanagan  and  his  co-workers  at  Lackland  Air  Force  Base  (AFB)  as  an  example  of  significant 
improvement.  Colonel  Flanagan  subsequently  used  similar  methods,  which  he  called  a  "Critical 
Incidents  Technique,"  to  analyze  administrative  officers’  and  research  and  development  officers’ 
jobs,  thus  creating  another  job  analysis  methodology. 

In  1946,  Shartle  called  for  establishment  of  a  national  center  for  job  analysis;  he  believed  an 
integrated  approach  was  essential  to  derive  proper  occupational  information  and  optimize  its  use 
(Shartle,  1959).  In  an  August  1949  memorandum  from  the  Joint  Chiefs  of  Staff  to  the  Secretary 
of  Defense,  a  request  was  made  to  initiate  a  study 

to  determine  the  most  appropriate  methodology  and  techniques  of  military  job 
analysis  and  job  evaluation;  determine  the  extent  to  which  the  unilateral  analyses 
accomplished  to  date  are  valid  with  relation  to  methodology;  provide  for  such 
further  job  analysis  as  may  be  required;  and  initiate  a  study  to  relate  all  Army, 

Navy,  and  Air  Force  jobs  to  a  common  occupational  structure. "  (Van  Cleve,  1974, 
p.  65) 

In  the  1950’s,  Rupe  (1952,  1956)  studied  several  Air  Force  job  analysis  methods  and 
recommended  adoption  of  a  checklist  survey  method;  this  suggestion  met  with  considerable 
resistance  even  though  several  members  of  the  Air  Staff  pleaded  for  an  aggressive  occupational 
research  program  (Christal,  1974a).  Finally,  in  late  1957,  Headquarters  United  States  Air  Force 
(HQ  USAF)  directed  that  an  occupational  research  project  be  established.  It  took  another  10 
years  to  develop  an  operational  Air  Force  occupational  analysis  program,  which  was  finally 
implemented  in  July  1967.  This  program  included  initial  operational  use  of  the  Comprehensive 
Occupational  Data  Analysis  Programs  (CODAP);  a  number  of  additional  years  were  required 
to  get  it  institutionalized  in  the  classification  and  training  communities  within  the  Air  Force 
(Christal,  1974b;  Mitchell,  1988;  Morsh,  1964). 

Throughout  the  1950’s  and  1960’s,  several  military  job  analysis  methodologies  continued  to  be 
used  by  the  various  services;  several  proved  difficult  to  use  or  too  expensive  to  sustain  (Van 
Cleve,  1974).  The  advantage  offered  by  the  Air  Force’s  task  inventory,  CODAP-based  program 
was  the  capability  to  obtain  and  analyze  detailed  task  information  from  large  numbers  of  job 
incumbents.  Of  this  work,  McCormick  (1967)  wrote: 
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The  pioneering  work  of  the  Personnel  Research  Laboratory... is  particularly 
noteworthy.  They  have  developed  and  used  job  inventories,  consisting  of  lists  of 
job  tasks  in  a  given  career  field,  as  a  means  of  mass  collection  of  job  information 
from  job  incumbents.  This  technique,  in  combination  with  certain  associated 
statistical  procedures  (which  they  have  also  developed),  has  made  it  possible  to 
describe  jobs  in  quantitative  terms  and  to  express  job  relationships  in  quantitative 
terms,  (p.  122) 


The  Task  Inventory/CODAP  Approach 

The  research  came  about  when  the  demands  of  managing  personnel  in  a  rapidly  changing,  high- 
technology  workplace  became  clear  (Rupe,  1956;  Thorndike,  Hagen,  Orr,  &  Rosner,  1957). 
These  demands  generally  fell  into  the  broad  categories  of  classification,  job  evaluation,  testing, 
and  training.  Early  in  this  line  of  research,  it  became  clear  that  a  detailed  understanding  of  Air 
Force  jobs  was  essential  to  all  these  goals  (Morsh,  Madden,  &  Christal,  1961).  At  the  same 
time,  the  potential  impact  of  the  electronic  digital  computer  was  being  recognized  (Christal, 
1974a;  Morsh  &  Christal,  1966).  Although  all  the  problem  areas  were  being  researched,  the 
first  gains  were  in  the  area  of  classification.  These  gains  were  due^  in  part,  to  two  major 
advances.  The  first  advance  was  the  validation  of  scientific  methods  for  developing 
standardized,  individual  job  descriptions  (Archer,  1966;  Archer  &  Fruchter,  1963).  The  second 
advance  was  the  development  of  an  analytic  tool,  hierarchical  clustering,  which  used  those 
descriptions  to  define  empirical  subdivisions  of  given  occupational  areas  (Ward,  1961). 

Early  studies  had  shown  Aat  the  best  information  for  classification  purposes  was  "time  spent" 
data  from  job  incumbents  (McCormick,  1960;  McCormick  &  Tombrink,  1960).  Many  of  these 
same  early  studies  also  examined  job  incumbents’  "self-report"  of  training  needs,  without  much 
success.  These  self-report  studies  included  analysis  of  task  "physical  difficulty"  and  "mental 
difficulty."  Other  early  studies  examined  self-reports  of  task  "training  emphasis,"  "difficulty 
to  learn  in  on-the-job  training  (OJT)"  (Morsh,  1965),  and  "worker-supervisor  agreement  on 
difficulty"  (Madden,  Hazel,  &  Christal,  1964).  Based  on  this  research,  it  was  decided  that 
training  requirements  data  would  best  be  collected  from  senior  supervisory  personnel. 

Task  Factor  Information  -  Task  Difficulty  Ratings 

Using  supervisory  raters,  several  scales  were  evaluated  and  subjected  to  interrater  agreement 
analyses.  It  became  clear  that  "difficulty"  was  multidimensional  and  that  refinement  of  that 
concept  was  required  in  order  to  develop  an  operational  system.  It  was  determined  that  a 
measure  of  "task  learning  difficulty"  would  best  serve  the  needs  of  the  training  community. 
First,  it  provided  a  stable,  usable  measure  that  could  be  collected  operationally.  Second,  this 
definition  of  difficulty  had  the  desirable  implication  of  relating  individual  learning  ability  to  an 
appropriate  training  course  length. 

Although  all  operational  data  collection  had  been  on  a  relative  scale  within  one  or  two  highly 
related  career  fields,  researchers  at  AFHRL  (Air  Force  Human  Resources  Laboratory-now 
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designated  the  Human  Resources  Directorate  of  Armstrong  Laboratory  or  AL/HR)  were  studying 
whether  these  data  could  be  used  across  many,  if  not  all,  career  areas  by  use  of  a  benchmark 
scale  (Burtch,  1978;  Fugill,  1972,  1973).  The  goal  of  the  benchmark  scale  research  was  to 
develop  measures  that  could  prioritize  Air  Force  career  fields  in  terms  of  learning  difficulty. 
In  addition,  this  measure  could,  on  a  standard  scale,  demonstrate  the  diversity  of  learning 
difficulty  requirements  within  a  career  field,  and  perhaps  suggest  classification  changes. 

Benchmark  scales  were  produced  for  the  major  aptitude  areas.  The  Armed  Forces  Vocational 
Aptitude  Battery  (ASVAB)  composites  for  Administrative,  General,  Electronics,  and  Mechanical 
were  the  four  aptitude  areas.  The  Administrative  and  General  areas  were  then  combined  to  yield 
three  major  areas.  Tasks  spanning  the  spectrum  of  within-specialty  difficulty  were  selected  for 
study.  Contractors  made  field  studies  of  the  selected  tasks  and  rated  them  on  the  appropriate 
25-point  benchmark  scale.  Analytic  software  was  developed  to  generate  regression  equations 
predicting  the  25-point  benchmark  ratings  from  the  corresponding  subset  of  the  original 
noncommissioned  officer  (NCO)  ratings  of  within-specialty  learning  difficulty.  The  resulting 
regression  equations  were  Aen  used  to  produce  predicted  25-point  ratings  for  all  tasks  within  the 
original  inventory.  Subsequently,  these  25-point  ratings  were  used  to  compute,  for  every  job 
incumbent  surveyed,  a  predicted  job  learning  difficulty  index  on  a  scale  of  10  to  250.  The  data 
were  summarized  for  the  first-term,  5-skill-level  airmen  within  the  specialty.  The  mean  and 
standard  deviation  for  this  group  have  become  known  as  the  Occupational  Learning  Difficulty 
(OLD)  index  for  the  specialty. 

The  OLD  indexes  have  been  produced  by  a  lengthy,  careful  research  and  development  process. 
However,  because  of  the  important  applications  in  which  these  OLD  indexes  would  be  used, 
several  validation  methodologies  were  also  employed.  The  Aptitude  Requirements  contractor 
ratings  (Burtch,  1978)  correlated  well  with  the  original,  within-specialty  ratings  of  difficulty. 
This  finding  increased  confidence  in  the  basic  data.  Subsequent  research  has  demonstrated  a 
construct  validity  with  respect  to  training  course  attributes  of  student  input,  training  outcomes, 
and  course  characteristics  (Weeks,  Mumford,  &  Harding,  1985).  Further  research  might  be 
undertaken  to  explore  the  actual  tradeoff  curves  in  the  difficulty-aptitude  interaction  in  order  to 
enhance  the  accuracy  of  the  person-job-match  algorithms. 

The  importance  of  the  construct  of  task  learning  difficulty  for  Air  Force  technical  training  is 
based  on  the  assumption  that  it  will  take  longer  to  learn  a  more  difficult  task  (Christal,  1974b; 
Weeks,  1981).  This  appears  to  be  a  well-founded  assumption  whose  corollary  that  the  more 
capable  individuals  will  take  less  time  to  learn  a  task  of  constant  difficulty  is  well  supported  in 
the  research  literature  (Cronbach  &  Snow,  1977).  The  recent  cognitive  literature  further 
supports  the  concept  that  "time  to  mastery"  and  "response  time"  may  be  excellent  indicators  of 
problem  difficulty,  both  at  the  individual  and  aggregate  levels  (Sternberg,  1982). 

In  a  more  recent  study  (Mumford,  Harding,  Fleishman,  &  Weeks,  1987),  the  construct  of 
"occupational  difficulty"  (the  aggregated  difficulties  of  tasks  performed  within  a  specialty)  was 
included  in  a  conceptual  model  of  the  Air  Force’s  Initial  Skills  Training  process  because 
Instructional  System  Design  personnel  and  training  managers  felt  that  task  difficulty  had  a 
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significant  influence  on  the  design  of  courses  and  on  student  performance.  Inspection  of  the  data 
collected  in  almost  40  training  courses  supported  these  a  priori  expectations;  that  is,  the  more 
technical  courses  had  higher  occupational  difficulty  indexes  than  did  less  technical  courses. 
Weeks  (1984)  performed  a  more  detailed  analysis  of  these  data,  in  which  the  pattern  of 
relationships  observed  between  the  occupational-level  index  and  several  training  criteria 
substantiated  the  validity  of  the  learning  difficulty  construct. 

The  results  of  a  path  analysis  carried  out  by  Mumford  et  al.  (1987)  involving  the  student  input, 
course,  and  training  outcome  variables  included  in  the  model  showed  that  occupational  difficulty 
was  a  prime  determinant  of  the  nature  of  the  training  course  by  influencing  course  attributes  such 
as  course  length,  student-faculty  ratio,  instructor  quality,  and  amount  of  feedback  provided. 
OLD  was  shown  to  have  a  negative  effect  on  the  quality  of  student  performance;  that  is,  the 
more  difficult  the  tasks  within  a  specialty,  the  poorer  the  quality  of  performance  in  training. 
These  findings  further  indicate  the  importance  of  task  difficulty  in  Air  Force  technical  training 
and  support  the  need  for  further  research  on  the  relationship  between  training  criteria  and  the 
task  learning  difficulty  index. 

Once  task  difficulty  ratings  for  specialties  have  been  placed  on  a  common  metric  (via  the 
"benchmarking"  process),  then  the  relative  difficulty  of  typical  jobs  of  all  specialties  can  be 
compared  in  order  to  develop  a  rank-ordering  of  the  specialties  themselves.  Management 
decisions  can  then  be  made  on  minimum  aptitudes  required  for  each  specialty  (Burtch, 
Lipscomb,  &  Wissman,  1982;  Fugill,  1972,  1973).  As  of  April  1982,  such  OLD  data  have  been 
used  to  operationally  redefine  such  aptitude  requirements,  and  current  standards  are  being  slowly 
adjusted  (5  percent  per  year)  to  meet  the  new  target  values  (Harding,  Mumford,  &  Weeks,  1985; 
Weeks,  1984).  The  use  of  task  difficulty  (TD)  and  OLD  data  impacts  on  the  total  distribution 
of  all  incoming  Air  Force  enlisted  personnel. 

Another  use  of  such  data  involves  the  Pre-Enlistment  Person- Job  Match  system  (Hendrix,  Ward, 
Pina,  &  Haney,  1979).  The  OLD  is  one  variable  included  in  the  Procurement  Management 
Information  System  (PROMIS),  which  allocates  new  personnel  to  the  available  jobs;  this 
computer-based  decision  system  is  used  interactively  to  offer  appropriate  jobs  to  qualified 
individuals.  The  most  difficult  jobs  are  offered  to  the  most  talented  enlistees  of  a  constrained, 
ordered  list.  Once  an  occupational  placement  is  made,  the  individual  can  be  entered  into  the 
Pipeline  Management  System  (PMS)  to  notify  the  Air  Force  Military  Personnel  Center  (AFMPC) 
and  Air  Training  Command  (ATC). 

Training  Emphasis  Data 

Another  task  factor  collected  from  senior  technicians  is  ratings  of  how  much  training  emphasis 
(TE)  should  be  given  to  a  task  in  Initial  Skills  Training;  that  is,  training  for  the  first  job  or  first 
enlistment  (Mitchell,  Ruck,  &  Driskill,  1988;  Ruck,  Thompson,  &  Stacy,  1987).  Senior  NCOs 
serving  in  operational  units  in  each  specialty  are  asked  (a)  to  check  each  task  for  which  formal 
training  of  some  kind  is  recommended  for  first-term  airmen,  and  (b)  to  rate  each  of  the  tasks 
checked  to  reflect  the  degree  of  recommended  emphasis  (Ruck  et  al.,  1987).  Vaughan  (1978) 
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noted  that  the  ratings  indicate  the  extent  to  which  emphasis  should  be  placed  on  each  task  in 
formal  training  for  first-assignment  airmen  (i.e.,  at  the  apprentice  level);  he  went  on  to  observe 
that  the  ratings  do  not  distinguish  among  the  various  forms  of  formal  training  (such  as  technical 
school,  correspondence  course,  field  training  unit,  and  on-the-job  training  or  OJT). 

When  used  in  conjunction  with  information  on  the  percentage  of  first-assignment  airmen 
performing  tasks,  TE  and  TD  data  can  be  used  to  evaluate  Plans  of  Instruction  for  basic  entry- 
level  courses,  and  other  training  specification  documents.  The  tasks  can  be  reorganized  into 
displays  structured  around  the  documents  being  evaluated,  with  task  data  grouped  into 
appropriate  categories  for  each  section  or  paragraph.  This  procedure  reformats  the  data  into  a 
form  that  is  familiar  to  the  field  technician,  training  developer,  or  training  decision-maker,  yet 
provides  concise  summaries  of  information  from  a  substantial  proportion  of  career  field 
incumbents  and  senior  technicians.  By  displaying  data  summaries  in  this  way,  a  direct  validation 
of  training  in  terms  of  tasks  performed  on  the  job  can  be  achieved.  Areas  of  overtraining  and 
undertraining  can  be  identified  for  review,  as  well  as  groups  of  tasks  that  may  have  been  omitted 
from  training  specification  documents. 

By  reviewing  such  summaries  of  occupational  analysis  data,  functional  and  training  managers 
can  direct  needed  modifications  to  training  documents  and,  ultimately,  to  training  programs. 
These  decisions  are  then  validated  by  meeting  with  Air  Force  functional  managers  and 
representatives  of  user  commands  who  employ  new  specialists.  This  extra  validation  step  is 
normally  conducted  in  a  Utilization  and  Training  Workshop  (U&TW),  and  agreemente  with  user 
organizations  represent  a  contract  as  to  the  level  of  training  to  be  provided  in  the  initial  skill 
acquisition  courses.  Implicit  in  this  kind  of  agreement  is  the  acceptance  by  user  organizations 
to  provide,  from  their  own  resources,  any  additional  training  (such  as  hands-on  training  and 
experience  with  equipment)  not  provided  in  the  classroom.  The  U&TWs  also  permit  functional 
managers  to  develop  future  organizational  plans,  such  as  the  impending  procurement  of  new 
systems  or  phase-out  of  obsolete  equipment.  Thus,  occupational  analysis  data  can  be  tempered 
by  both  user  judgments  and  future  plans  for  changes  in  the  technical  specialty. 

This  TE  line  of  research  has  resulted  in  a  variety  of  other  research  and  development  (R&D) 
programs  for  the  prioritization  of  training,  assistance  of  training  decision-makers,  and 
restructuring  of  occupations  or  specialties.  The  Training  Decisions  Modeling  Technology  (see 
Chin,  Lamb,  Bennett,  &  Vaughan,  1992;  Mitchell,  Yadrick,  &  Bennett,  1993)  is  based  on  the 
need  to  have  an  overall  model  of  each  specialty’s  jobs  and  training  programs;  any  change  in  one 
job  or  program  impacts  on  all  parts  of  the  Air  Force  specialty  (AFS)  in  terms  of  training  costs 
or  capacity.  This  R&D  area  is  too  complex  to  detail  here,  but  some  of  the  new  applications  are 
covered  in  other  sections  of  this  symposium.  Suffice  it  to  say  that  for  the  Air  Force,  the  use 
of  TE  and  TD  ratings  has  led  to  a  variety  of  R&D  applications  which  have  substantially 
enhanced  the  ability  of  the  Air  Force  to  provide  relevant  and  cost-effective  training  programs. 
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Other  Types  of  Task  Factor  Information 

A  number  of  other  efforts  have  been  made  to  address  task,  knowledge,  or  skill  requirements  of 
various  occupations.  The  Canadian  National  Defence  Forces  collect  ratings  from  job  incumbents 
on  the  levels  to  which  incumbents  are  involved  with  the  tasks  of  their  occupations  (assist, 
perform,  teach,  supervise,  etc.),  as  well  as  knowledges  required.  The  U.  S.  Navy  collects 
training  importance  ratings  from  senior  technicians  in  selected  jobs  in  order  to  evaluate  specific 
course  content;  the  Navy  also  surveys  supervisors  of  new  graduates  to  evaluate  training 
effectiveness  for  job  performance  (Ruck  &  L^g,  1986).  The  U.  S.  Army  collects  a  variety  of 
task  ratings  (task  difficulty,  training  emphasis,  training  importance,  part-of-the-job,  real  time, 
etc.)  on  skills  and  knowledges  when  requested  by  the  proponent  technical  training  centers  to 
provide  data  for  use  in  training  evaluations  or  selection  for  various  specialties  (Goldman,  1991a). 
Other  task  factors  or  factor-weighting  schemes  are  currently  undergoing  research  and 
development  (Goldman,  1991b). 

Gradually  Increasing  Acceptance 

The  work  of  Christal  and  his  co-workers  through  the  years  has  not  been  an  easy  achievement, 
nor  was  acceptance  immediate  or  widespread.  Job  and  occupational  analysis  is  not  a  glamorous 
or  high  visibility  area  on  which  to  build  a  personal  career  or  secure  tenure.  Bill  Cunningham 
(1989,  as  cited  in  Harvey,  1991)  has  observed  that  it  is  difficult  to  get  students  interested  or  to 
get  reports  accepted  for  publication;  he  once  characterized  the  job  analysis  area  as  "the  Rodney 
Dangerfield  of  I/O  [Industrial/Organizational]  psychology:  it  doesn’t  get  a  lot  of  respect"  (p.  73). 
The  R&D  community  as  a  whole  (across  several  academic  disciplines)  has  generally  not  been 
enthusiastic  about  the  largely  atheoretical  job  analysis  area. 

Yet  the  CODAP  approach  to  job  analysis  has  gradually  gained  wide  acceptance  within  the 
military  services  and  in  allied  services  (Mitchell,  1988).  Its  use  has  spread  to  the  academic 
world,  to  business  and  industry,  and  to  state  and  local  government  (see  Figure  1).  Harry 
Ammerman  and  his  co-workers  at  the  National  Center  for  Research  in  Vocational  Education  at 
The  Ohio  State  University  created  a  central  library  of  task  lists,  as  well  as  publishing  a  five- 
volume  set  of  booklets  on  the  task  inventory  approach  for  determining  performance  content  for 
job  training  (Ammerman  &  Pratzner,  1977).  This  was  a  major  step  forward  in  civilian 
application  of  the  task-based  approach  to  occupational  analysis. 

CODAP  software  has  recently  been  upgraded  and  standardized  across  the  U.  S.  and  allied 
military  services  (see  Chapter  II).  CODAP  can  also  interface  with  other  software  systems; 
CODAP  files  can  be  extracted  and  exported  to  general  statistical  packages,  such  as  the  Statistical 
Package  for  the  Social  Sciences  (SPSS)  or  SAS.  Likewise,  other  types  of  data  (such  as  Critical 
Incidents;  Skills,  Knowledges,  and  Ability  Requirements;  etc.)  can  and  are  being  processed  in 
CODAP  (see  Driskill,  Weissmuller,  Hageman,  &  Barrett,  1989;  Moon  et  al.,  1991).  Data  files 
are  routinely  exchanged  among  computers  via  tape  or  transmission  networks,  as  well  as  among 
hardware  systems.  A  microcomputer  version  called  "atCODAP"  has  been  created  by  Johnny 
Weissmuller  and  Sensible  Systems,  Inc.  and  is  being  used  by  a  variety  of  governmental  agencies 
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Figure  1.  Transfer  of  CODAP  technology  to  other  military  and  civilian  organizations. 


and  other  enterprises  (see  Figure  2).  This  system  includes  automatic  task  inventory 
development,  as  well  as  computer-based  survey  administration  and  disk  storage  and  transmittal 
of  survey  responses.  A  computer-based  task  inventory  administration  system  is  currently  under 
development  for  the  Air  Force  occupational  analysis  program. 

As  the  Task  Inventory/CODAP  (TI/CODAP)  approach  has  grown  and  matured,  so  have  other 
job  analysis  methodologies.  Among  the  more  notable  developments-some  of  which  have  been 
used  in  both  military  and  civilian  applications— are  Fleishman’s  Ability  Requirement  Scales 
(ARS;  Fleishman  &  Quaintance,  1984),  Primoff  s  Job  Element  Method  (Primoff  &  Fine,  1988), 
Fine’s  Functional  Job  Analysis  (Fine,  1988),  McCormick’s  Position  Analysis  Questionnaire 
(McCormick,  1979),  and  Cunningham’s  Occupational  Analysis  Inventory  (Cunningham,  1988). 
A  comprehensive  review  of  the  almost  half  a  hundred  methodologies  appearing  in  the  literature 
is  provided  in  Gael  (1988). 
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Figure  2.  Users  of  atCODAP  technology  and  services. 


Evaluations  of  Military  Occupational  Analysis 

Brian  Moore,  in  a  review  of  Job  analysis  methodologies  for  the  Navy,  viewed  modern  task 
inventories  as  a  synthesis  of  three  differing  approaches  to  job  design:  the  engineering  approach, 
the  job  content  tq)proach,  and  the  role  content  approach  (Moore,  1976).  Moore  concluded  his 
review  with  the  comment  that  "the  relevance,  utility,  and  comprehensiveness  of  the  TI/CODAP 
seems  to  offer  a  significant  step  forward  over  other  forms  of  occupational  analysis"  (p.  75). 
McCormick  (1979)  disagreed  with  Moore’s  assessment,  although  he  conceded  that  task  inven¬ 
tories  were  particularly  useful  for  identifying  job  types  and  for  planning  training  programs. 
McCormick  went  on  to  point  out  that  "each  of  the  various  individual  methods  has  its  own  poten¬ 
tial  uses,  and  it  does  not  seem  reasonable  to  believe  that  there  is  one  best  method  that  can  serve 
all  purposes"  (p.  135). 

Levine,  Thomas,  and  Sistrunk  (1988)  evaluated  a  number  of  job  analysis  methodologies  to 
determine  which  systems  are  most  useful  for  various  purposes.  They  utilized  data  from  a 
nationwide  survey  of  93  leading  job/occupational  analysts,  who  rated  the  characteristics  and 
utility  of  various  established  systems  including  McCormick’s  Position  Analysis  Questionnaire 
(PAQ),  Fine’s  Functional  Job  Analysis  (FJA),  TI/CODAP,  Flanagan’s  Critical  Incidents 
Technique  (CIT),  Primoff  s  Job  Element  Method  (JEM),  Fleishman’s  Ability  Requirement  Scales 
(ARS),  and  Lopez’s  Threshold  Traits  Analysis  (TTA).  The  PAQ,  FJA,  and  TI/CODAP  were 
generally  rated  among  the  top  three  methods  for  most  uses.  In  addition,  80  of  the  93 
respondents  indicated  that  they  preferred  to  use  a  combination  of  two  methods  to  meet  their 
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needs  (Levine  et  al.,  1988).  The  more  popular  combinations  included  TI/CODAP  and  PAQ, 
CIT  and  FJA,  FJA  and  TI/CODAP,  PAC  and  CIT,  and  TI/CODAP  with  ARS.  This  survey 
provided  a  high  level  of  visibility  for  TI/CODAP  for  the  first  time  to  academicians,  human 
resource  managers,  and  other  job  analysis  practitioners. 

It  should  be  noted  that  at  least  three  of  these  methods  (TI/CODAP,  PAQ,  and  CIT)  evolved 
directly  from  military  job  analysis  research;  and  at  least  two  others  (JEM  and  FJA),  from  other 
government  agencies.  Other  researchers,  such  as  Fleishman,  worked  closely  with  and  were 
often  supported  by  military  R«&D  agencies.  The  point  is  that  a  very  substantial  proportion  of 
job  analysis  research  in  this  country  has  been  directly  accomplished  by,  or  directly  or  indirectly 
supported  by,  the  military  services  and  other  government  agencies. 

Current  Status  and  Trends 

TI/CODAP  continues  to  be  the  primary  job  analytic  method  in  the  military.  Some  current 
developments,  however,  include  the  use  of  TI/CODAP  in  conjunction  with  other  methods  and 
approaches  (Driskill  et  al.,  1989)  to  address  a  variety  of  issues.  Because^ other  approaches 
address  different  kinds  of  task-job-worker  dimensions,  their  use  supplements  the  highly  specific 
task  information  provided  by  TI/CODAP  and  can  provide  vital  information  not  available  from 
TI/CODAP. 

Job  Analytic  Approaches 

Differences  among  job  analytic  approaches  lie  primarily  in  their  focus.  Fleishman  and 
Quaintance  (1984)  have  suggested  that  methods  can  be  classified  into  four  categories:  Behavior 
Description,  Behavior  Requirements,  Ability  Requirements,  and  Task  Characteristics. 

Behavior  Description.  Behavior  description  methods  focus  on  a  specification  of  what 
workers  actually  do  while  performing  their  jobs.  Products  of  these  methods  range  from  highly 
task-specific  to  more  general  task  information.  Usually,  emphasis  is  placed  primarily  on  the 
overt  behaviors  or  certain  subjective  terms  which  describe  what  the  worker  does.  Several 
systems  of  job  analysis  representing  this  approach  are  in  use,  each  differing  from  the  others  with 
respect  to  the  level  and  context  of  the  descriptive  information.  Included  under  this  approach  are 
TI/CODAP,  PAQ,  the  Occupational  Analysis  Inventory  (OAI),  the  General  Work  Inventory 
(Cunningham,  Wimpee,  &  Ballentine,  1990),  JEM,  and  FJA.  These  methods  are  well 
documented  with  regard  to  measurement  and  procedural  issues,  interrater  reliability  and 
taxonomic  adequacy,  job  component  and  criterion-related  validity,  and  utility  (Driskill  et  al., 
1989). 

Behavior  Requirements.  Behavior  requirements  methods  place  emphasis  on  the  classification 
of  behaviors  that  should  be  emitted  or  which  are  assumed  to  be  required  for  achieving  criterion 
levels  of  performance.  The  human  performer  is  assumed  to  be  in  possession  of  a  large  number 
of  processes  that  serve  to  intervene  between  the  stimulus  and  response  events.  These  intervening 
processes  are,  in  a  real  sense,  constructs  to  account  for  human  task  behavior.  Such  approaches. 
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best  described  by  the  methods  of  Gagne  (1977)  and  Miller  (1973),  have  most  generally  been 
applied  to  the  problem  of  human  learning. 

Ability  Requirements.  Ability  requirements  methods  (best  exemplified  by  Fleishman’s  ARS) 
describe,  contrast,  and  compare  jobs  and  tasks  in  terms  of  abilities  required  of  the  worker. 
"Abilities"  as  defined  by  Fleishman  and  Mumford  (1988)  and  McCormick  (1979)  are  the 
relatively  enduring  attributes  of  the  individual  performing  the  tasks,  and  certain  tasks  are 
hypothesized  as  requiring  certain  ability  profiles  if  performance  is  to  be  maximized.  The  ARS 
consists  of  a  taxonomy  of  sensory,  perceptual,  cognitive,  and  physical  abilities  which  raters  use 
to  describe  human  task  performance  requirements. 

Task  Characteristics.  Task  characteristics  methods  emphasize  the  characteristics  of  the  tasks 
that  are  performed.  Tasks  are  analyzed  in  terms  of  their  characteristics  and  their  impacts  on 
behavior  and  performance.  A  taxonomy  of  task  characteristics  includes  such  features  as  (a) 
number  of  procedural  steps,  (b)  the  dependency  of  procedural  steps,  (c)  response  rate,  (d) 
procedural  complexity,  and  (e)  amount  of  feedback  provided  the  performer  by  successful  (or 
unsuccessful)  outcomes  (Driskill  et  al.,  1989).  Use  of  the  task  characteristics  approach  has 
produced  important  implications  for  learning  and  retention.  A  study  in  which  hands-on 
performance  measures  were  the  criteria  (Rose  et  al.,  1984)  suggests  that  the  characteristics  of 
tasks  may  be  effective  predictors  of  learning  rates.  Based  on  the  kinds  of  descriptors  used,  one 
might  hypothesize  that  learning  difficulty  ratings  collected  at  the  task  level,  as  in  the  TI/CODAP 
method,  may  be  effective  learning  and  retention  predictors. 

Recent  Trends 

In  a  more  recent  effort,  Driskill  and  his  co-workers  (Driskill  et  al.,  1989)  examined  36  various 
job  analytic  methods  to  determine  how  ability  requirements  could  best  be  linked  to  task-level  job 
data.  As  a  result,  a  combination  of  TI/CODAP  and  the  ARS  was  successfully  employed. 
Another  application  demonstrated  how  knowledge  and  skill  taxonomies  can  be  used  with 
CODAP-generated  task  co-performance  modules  to  define  training  and  testing  requirements 
empirically  (Moon  et  al.,  1991).  Linkages  of  this  kind  are  essential  if  we  are  to  have  legally 
defensible  occupational  requirements,  realistic  recruiting  standards,  optimal  classification  and 
placement,  and  realistic  training  programs. 

Such  linkages  are  possible  only  if  current  methods  are  extended  and  a  variety  of  methods  are 
utilized.  Recent  and  on-going  research  is  addressing  the  use  of  multiple  approaches,  including 
the  ARS  and  the  Critical  Incidents  Technique,  in  conjunction  with  CODAP.  Further,  the 
analytic  capabilities  of  CODAP  are  being  extended  and  applied  to  practical  personnel  and 
training  issues. 

In  particular,  the  task  co-performance  module  technology  is  extremely  promising  for  defining 
classification  structures  that  optimize  transfer  of  training  and  for  the  empirical  determination  of 
skill  and  knowledge  requirements.  Task  modules  have  also  been  used  extensively  as  a  basis  for 
modeling  specialty  jobs  and  training  requirements  (see  Chin  et  al.,  1992),  and  have  high 
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potential  utility  in  multilevel  organizational  research  (Mitchell,  Phalen,  &  Hand,  1992;  Vaughan 
&  Yadrick,  1992).  In  one  recent  experimental  study,  ratings  of  task  modules  on  26  skill  and 
knowledge  scales  proved  predictive  of  learning  curves  (training  time  functions);  this  approach 
has  high  potential  for  estimating  training  requirements  of  new  weapon  systems  (Vaughan, 
Mitchell,  Knight,  Bennett,  &  Buckenmyer,  1990). 

Task  modules,  particularly  mission  essential  modules  (MEMs),  have  recently  been  proposed  as 
an  appropriate  level  of  analysis  for  a  specialty  structuring  system,  and  for  a  variety  of  other 
purposes.  Co-performance  modules  have  also  been  successfully  used  in  non-DoD  (Department 
of  Defense)  research  (see  Moon  et  al.,  1991)  and  appear  to  have  a  number  of  realistic  and 
practical  applications. 

A  number  of  recent  CODAP-related  research  and  applications  efforts  have  been  undertaken  for 
non-military  organizations  and  business  firms,  and  we  expect  this  trend  to  continue.  There  have 
been  several  uses  of  the  new  MODULE  technology  with  civilian  companies  and  non-DoD 
agencies  and  organizations,  and  there  appears  to  be  considerable  interest  in  making  further 
applications  of  military-developed  job  analysis  technology  for  civilian  use.  This  is  a  trend  we 
should  encourage  and  foster;  we  need  to  find  ready  application  of  such  technologies  in  the 
civilian  sector  if  we  hope  to  be  able  to  justify  further  R&D  in  the  future.  The  U.  S.  military 
services  can  no  longer  afford  to  be  the  primary  source  of  funding  for  this  very  significant  area. 
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n.  INNOVATIONS  IN  OCCUPATIONAL  MEASUREMENT  TECHNOLOGY 

FOR  THE  U.  S.  MILITARY 


William  J.  Phalen 
Jimmy  L.  Mitchell 


The  principal  occupational  analysis  technology  in  the  United  States  Air  Force  is  the  Task 
Inventory/Comprehensive  Occupational  Data  Analysis  Programs  (TI/CODAP)  approach.  This 
system  has  supported  a  major  occupational  research  program  within  the  Air  Force  Human 
Resources  Laboratory  (AFHRL;  now  the  Armstrong  Laboratory,  Human  Resources  Directorate) 
since  1962  (Christal,  1974b;  Morsh,  1964),  and  an  operational  occupational  analysis  capability 
within  Air  Training  Command’s  USAF  Occupational  Measurement  Squadron  since  1967 
(Driskill,  Mitchell,  &  Tartell,  1980;  Weissmuller,  Tartell,  &  Phalen,  1988).  The  CODAP 
system  is  now  used  by  all  the  U.  S.  and  many  allied  military  services,  as  well  as  a  number  of 
other  government  agencies,  academic  institutions,  and  some  private  industries  (Christal  & 
Weissmuller,  1988;  Mitchell,  1988). 

The  TI/CODAP  approach  to  job  analysis  is  generally  recognized  as  being  particularly  relevant 
for  a  number  of  human  resource  management  (HRM)  uses  including  job  descriptions, 
identification  of  training  requirements,  and  modeling  career  paths  (McCormick,  1976).  In  the 
early  1980’s,  a  survey  of  100  leading  U.S.  job  analysts  evaluated  six  major  job  analysis  systems 
on  a  number  of  utility  dimensions,  such  as  cost  and  time  to  complete,  and  for  a  variety  of  HRM 
uses  such  as  evaluation,  performance  appraisal,  and  training  (Levine,  Ash,  Hall,  &  Sistrunk, 
1983).  The  TI/CODAP  approach  to  job  analysis,  along  with  McCormick’s  Position  Analysis 
Questionnaire  (PAQ;  McCormick,  Jeanneret,  &  Mecham,  1972)  and  Sidney  Fine’s  Functional 
Job  Analysis  (FJA;  Fine  &  Wiley,  1971),  were  consistently  rated  among  the  top  three  for  most 
HRM  uses  (Levine  et  al.,  1983).  Significantly,  three  of  the  six  major  job  analysis  systems  in 
the  U.  S.  (Critical  Incidents,  PAQ,  and  TI/CODAP)  were  developed  in  military-funded  job 
analysis  research;  such  military-sponsored  R&D  has  had  a  substantial  and  lasting  impact  on  the 
study  of  jobs  in  this  country  (Mitchell,  1988). 

In  assessing  the  current  state  of  job  analysis  technology  in  America,  Harvey  (1991)  has  recently 
noted: 


Advances  in  computer  technology  have  allowed  the  development  of  integrated 
personnel  systems  to  manage  the  vast  amounts  of  data  generated  during  the  job 
analysis  process.  Applications  of  artificial  intelligence  and  expert-systems 
technology  promise  to  ftirther  reduce  the  cost  and  labor-intensiveness  of  the  job 
analysis  process,  (p.  71) 

A  good  bit  of  this  work  has  already  been  done,  is  in  progress,  or  is  planned  for  the  near  future 
by  the  military  occupational  analysis  community. 
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In  recent  years,  the  CODAP  system  has  been  rewritten  to  make  it  more  efficient  and  to  expand 
and  partially  automate  its  capabilities  (Phalen,  Mitchell,  &  Staley,  1987).  In  the  process  of 
developing  this  new  ASCII  CODAP  system,  several  major  innovative  programs  were  created  to 
extend  the  capabilities  of  the  system  for  assisting  analysts  in  identifying  and  interpreting 
potentially  significant  jobs  (groups  of  similar  cases)  and  task  modules  (groups  of  co-performed 
tasks). 

Over  the  last  several  years,  operational  testing  and  evaluation  of  new  ASCII  CODAP  interpretive 
software  has  continued,  and  these  programs  have  demonstrated  their  value  in  terms  of  enhanced 
analytic  capabilities  and  their  potential  to  accelerate  completion  of  an  occupational  analysis. 

A  Suite  of  Advanced  Interpretive  Assistance  Programs 

A  set  of  seven  programs  has  been  developed  to  assist  analysts  in  interpreting  job  and  task 
clusters.  Some  of  these  programs  were  completed  in  time  to  be  released  with  the  initial  version 
of  ASCII  CODAP;  others  required  further  refinement  before  they  were  ready  for  operational 
use.  It  is  helpful  to  have  an  overview  of  the  entire  set  of  programs,  so  everyone  can  see  how 
the  programs  relate  to  one  another  and  to  their  ultimate  objective.  These  programs  are  shown 
in  Figure  3. 
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Figure  3.  The  Set  of  Advanced  Interpretive  Assistance  Programs. 


Program  Descriptions 

These  programs  may  be  briefly  described  as  follows: 

.TOBTYP.  The  JOBTYP  program  automatically  identifies  stages  in  most  branches  of  a 
hierarchical  clustering  DIAGRM  which  represent  the  "best"  candidates  for  job  types.  First,  core 
task  homogeneity,  task  discrimination,  a  group  size  weight,  and  a  loss  in  "between"  overlap  for 
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merging  stages  are  calculated  for  all  stages  and  these  values  are  used  to  compute  an  initial 
evaluation  value  (for  JOBTYP  equations,  see  Haynes,  1989).  The  value  is  used  to  pick  three 
sets  of  initial  stages;  these  are  then  inserted  into  a  super /subgroup  matrix  for  additional  pairwise 
evaluation,  in  order  to  further  refine  the  selection  of  candidate  job  type  groups.  Three  final  sets 
of  stages  (primary,  secondary,  and  tertiary  groups)  are  then  reported  for  the  analyst  to  use  as 
starting  points  for  selecting  final  job  types. 

CORTAS.  The  CORTAS  program  compares  a  set  of  group  job  descriptions  ("contextual" 
groups)  in  terms  of  number  of  core  tasks  performed,  percent  members  performing  and  time 
spent  on  each  core  task,  and  the  ability  of  each  core  task  to  discriminate  each  group  from  all 
other  groups  in  the  set  (Phalen  &  Weissmuller,  1981).  It  also  computes  for  each  group  an 
overall  measure  of  within-group  overlap  called  the  "core  task  homogeneity  index, "  an  overall 
measure  of  between-group  difference  called  the  "index  of  average  core  task  discrimination  per 
unit  of  core  task  homogeneity,"  and  an  asymmetric  measure  of  the  extent  to  which  each  group 
in  the  set  qualifies  as  a  subgroup  or  supergroup  of  every  other  group  in  the  set. 

CASSET.  The  CASSET  program  generates  displays  of  cases  whose  jobs  are  most 
representative  of  job  types  within  a  given  set  of  job  clusters.  This  approach  permits  an  analyst 
to  quickly  characterize  a  job  type  by  the  salient  features  of  its  most  representative  and 
discriminating  members.  The  CASSET  report  may  contain  any  type  of  background  variable 
information  describing  a  case  that  will  fit  in  the  allocated  space.  "Base  of  assignment"  and  "job 
title"  are  often  the  most  useful  variables  to  aid  analysts’  interpretations. 

MODTYP.  Just  as  the  JOBTYP  program  automatically  selects  from  a  hierarchical  clustering 
of  cases  the  best  set  of  job  types,  based  on  similarity  of  time  spent  across  tasks,  the  experimental 
MODTYP  (module  typing)  program  selects  from  a  hierarchical  clustering  of  tasks  the  best  set 
of  task  module  types,  based  on  task  co-performance  across  cases.  The  term  "best"  means  that 
the  evaluation  algorithm  initially  optimizes  on  four  criteria  simultaneously  (i.e.,  within-group 
homogeneity,  between-group  discrimination,  group  size,  and  drop  in  "between  overlap"  in 
consecutive  stages  of  the  hierarchical  clustering).  After  all  stages  of  the  clustering  have  been 
evaluated  on  these  criteria,  primary  groups  are  input  to  the  TASSET  and  CORCAS  programs 
to  provide  analytic  and  interpretive  data  for  each  task  cluster  (Phalen,  Staley,  &  Mitchell,  1989). 
Task  clustering  using  co-performance  values  as  a  basis  for  developing  task  modules  (TMs)  has 
been  reported  elsewhere  and  need  not  be  detailed  again  here  (Mitchell,  Phalen,  &  Hand,  1991; 
Perrin,  Knight,  Mitchell,  Vaughan,  &  Yadrick,  1988;  Vaughan  et  al.,  1989).  Task  co¬ 
performance  is  defined  as  a  measure  of  the  similarity  of  pairs  of  task  profiles  across  all  the 
people  in  an  occupational  survey  sample.  Details  of  the  computation  of  measures  of  task  co¬ 
performance  may  be  found  in  Rue,  Rogers,  and  Phalen  (1992). 

TASSET.  TASSET  is  a  program  that  compares  clusters  of  tasks  (modules)  in  terms  of  the 
degree  to  which  each  cluster  of  tasks  is  co-performed  with  every  other  task  cluster  (supergroup/ 
subgroup  matrix).  TASSET  computes  the  average  co-performance  of  each  task  with  every  other 
task  in  each  cluster  (representativeness  index)  and  the  difference  in  average  co-performance  of 
the  same  tasks  with  all  other  task  clusters  (discrimination  index).  TASSET  also  identifies  tasks 
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that  meet  the  co-performance  criterion  for  inclusion  in  clusters  in  which  they  were  not  placed 
(potential  core  tasks),  as  well  as  tasks  that  are  highly  co-performed  with  clusters  other  than  the 
target  cluster  (negatively  unique  tasks). 

CORCAS.  The  CORCAS  program  characterizes  a  task  cluster  (module)  in  terms  of  the 
people  who  most  perform  the  tasks  in  the  cluster,  and  especially  those  principal  performers 
whose  jobs  are  concentrated  in  this  task  cluster  to  the  exclusion  of  all  or  most  other  task 
clusters.  The  CORCAS  report  may  contain  any  type  of  background  variable  information 
describing  a  case  that  will  be  useful  to  the  analyst  in  interpreting  the  task  cluster  and  which  will 
fit  in  the  allocated  space,  just  as  on  a  PRTVAR  report;  however,  "base  of  assignment"  and  "job 
title"  are  often  the  most  useful  variables. 

JOBMOD.  The  JOBMOD  (job  type  versus  task  module  mapping)  program  aggregates  the 
case-  and  task-level  indexes  computed  by  the  four  advanced  analysis  programs  and  uses  these 
aggregate  measures  to  relate  task  clusters  to  job  types  and  vice  versa.  The  description  of  job 
types  by  a  handful  of  discriminant  clusters  of  tasks,  and  the  association  of  each  task  cluster  with 
the  types  of  jobs  of  which  it  is  an  important  component,  is  a  basic  requirement  for  defining  and 
integrating  the  manpower,  personnel,  and  training  (MPT)  components  of  an  existing  or  potential 
Air  Force  specialty  (AFS)  or  weapon  system.  If  AFSs  are  to  be  collapsed  or  shredded  out,  or 
new  jobs  are  to  be  assigned  to  an  occupational  area,  or  old  jobs  are  to  be  moved  to  another 
occupational  area,  such  highly  summarized  yet  meaningfully  discriminant  hard  data  are  essential 
(Phalen  et  al.,  1989). 

Discussion 

The  work  on  CODAP  programs  for  selecting  and  interpreting  task  clusters  has  been  highly 
successful,  and  we  have  only  begun  to  tap  the  potential  of  this  type  of  automated  modular 
technology.  Further,  our  work  with  TMs  to  date  has  led  us  to  believe  that  the  task  module 
approach  has  real  promise  for  simplifying  and  expanding  the  use  of  occupational  information  in 
helping  executives  and  managers  make  more  realistic  decisions.  It  is,  in  fact,  a  critical 
technology  in  the  current  period  of  manpower  and  budget  reductions  and  consolidations.  It  is 
more  important  than  ever  to  be  able  to  model  proposed  changes  and  assess  the  potential  impact 
of  such  changes  before  final  MPT  decisions  are  made. 

We  believe  that  the  automated  modular  technology  can  be  extremely  useful  in  modeling 
occupations  and  the  world  of  work  (Mitchell,  Phalen,  &  Hand,  1992;  Weissmuller,  1978).  With 
additional  refinement,  we  should  be  able  to  use  quite  a  variety  of  information  to  develop  better 
TMs  which  will  take  into  account  the  type  of  equipment  operated  or  maintained,  as  well  as 
subject-matter  expert  (SME)  ratings  such  as  training  emphasis  (TE)  and  task  difficulty  (TD). 
Though  not  yet  fully  explored  or  validated,  this  emerging  TM  development  methodology  has 
great  promise  for  significant  improvement  of  military  manpower,  personnel,  and  training 
planning  and  decision-making,  and,  indeed,  for  organizational  analysis  as  well.  These  are,  of 
course,  the  first  steps  toward  a  fully  automated  interpretive  (artificial  intelligence  or  AI)  system. 
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We  have  detailed  the  development  and  operational  testing  of  the  new  CODAP  analyst  assistance 
programs  at  previous  International  Occupational  Analysts  Workshops  (Mitchell,  Hand,  & 
Phalen,1991;  Phalen,  Mitchell,  Staley,  1987;  Phalen,  Staley,  &  Mitchell,  1987)  and  at  Military 
Testing  Association  annual  meetings  (Mitchell  &  Phalen,  1985;  Mitchell,  Phalen,  &  Hand,  1991, 
1992;  Mitchell,  Phalen,  Haynes,  &  Hand,  1988;  Phalen,  Mitchell,  &  Hand,  1990;  Phalen, 
Staley,  &  Mitchell,  1988).  Indeed,  such  conferences  have  become  the  primary  forums  for 
interaction  among  CODAP  practitioners. 

New  Work  In  Progress 

There  are  a  number  of  projects  currently  underway  to  further  improve  the  TI/CODAP 
technologies  in  terms  of  better  data  collection,  improved  scales,  and  the  capability  to  link  task 
data  with  equipment  or  systems  operated  or  maintained,  required  skills  and  knowledges,  and 
other  relevant  dimensions. 

Computer-Based  Survey  Technology 

One  of  the  most  innovative  current  projects  is  the  Automated  Survey /Tailored  Task  List  system, 
a  computer-based  occupational  survey  administration  system  being  developed  to  meet  a  request 
from  the  USAF  Occupational  Measurement  Squadron  and  Air  Training  Command.  The 
capability  to  tailor  task  lists,  needed  for  inventory  development,  also  meets  a  requirement  of  the 
Base  Training  System  project  in  the  Human  Systems  Center/Systems  Program  Office  (HSC  SPO) 
to  be  able  to  integrate  task  lists  (as  specialties  are  merged,  as  in  RIVET  WORKFORCE). 

The  objective  of  the  project  is  to  develop  automated  procedures  that  provide  for  electronic 
distribution  of  task-level  surveys  to  respondents,  automated  self-administration  of  the  surveys 
on  PCs,  and  the  flow  of  electronically  captured  response  data  to  USAFOMS  on  communications 
lines  such  as  the  Military  Data  Network  (MILNET).  This  paperless  survey  administration 
system  will  result  in  substantial  reductions  in  costs  and  the  time  it  takes  to  collect  required 
survey  data.  The  more  timely  collection,  analysis,  and  reporting  of  data  should  benefit  all  users 
of  occupational  analysis  data. 

An  important  aspect  of  the  R&D  has  been  the  development  and  testing  of  five  scaling  techniques 
for  measuring  time  spent  on  a  task: 

1.  a  criterion  set  of  scales  that  provide  absolute  measures  of  frequency  of  task  performance, 
time  it  takes  for  a  single  performance  of  the  task,  and  total  amount  of  task  time  spent  per  week, 
month,  and  year  on  the  task; 

2.  a  three-stage  scaling  technique  that  begins  with  administration  of  the  usual  9-point  relative 
time  spent  scale  (stage  one),  proceeds  to  a  refinement  phase  in  which  tasks  assigned  the  same 
rating  are  displayed  together  and  moved  to  another  rating  category  if  appropriate  (stage  two), 
and  ends  with  a  phase  that  offers  the  opportunity  to  further  subdivide  the  tasks  within  each  rating 
category  into  two  or  three  subcategories  (stage  three); 
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3.  an  end-anchored  graphical  scale  that  displays  a  horizontal  line,  80  characters  long,  with 
the  individual’s  previously  determined  highest  and  lowest  time  spent  tasks  displayed  at  each  end 
of  the  scale  (as  each  task  is  presented,  the  rating  is  made  by  moving  the  cursor  along  the  line 
to  the  desired  distance  between  the  highest  and  lowest  time  spent  tasks); 

4.  a  direct-magnitude  estimation  scale  that  uses  a  previously  identified  "moderate"  time  spent 
task  as  the  anchor  with  an  assigned  value  of  100,  and  each  task  presented  for  rating  is  assigned 
a  value  relative  to  the  anchor  task  (e.g.,  if  the  task  to  be  rated  is  twice  as  time-consuming  as  the 
anchor  task,  it  should  be  assigned  a  value  of  200;  if  half  as  time-consuming,  50);  and 

5.  an  indirect-magnitude  estimation  scale  that  consists  of  nine  verbal  expressions  of  amount 
of  time  spent,  such  as  "very  little, "  "fairly  much, "  "a  great  amount, "  which  have  been  weighted 
by  means  of  magnitude  scaling  as  to  the  amount  of  time  each  expression  represents  compared 
to  the  term  "some. "  The  rater  responds  by  highlighting  the  appropriate  expression. 

On  each  scale,  multiple  feedback  loops  have  been  provided  to  solicit  respondent  evaluation  and 
refinement  of  responses.  When  all  task  responses  have  been  compiled,  a  complete  description 
of  the  job  is  presented  to  the  incumbent  for  final  review  and  editing.  This  description  gives  the 
estimated  hours  per  week  and  per  month  spent  on  each  task  by  the  respondent.  These  estimates 
were  made  possible  by  linking  several  of  the  scale  responses  to  the  absolute  measures  provided 
by  the  absolute  time  estimation  scale  and  averaging  multiple  linear  fits  to  the  experimental  scale 
data.  This  technology  promises  to  give  much  better  estimates  of  "real  time  expended  per  task" 
than  have  previously  been  possible  in  the  pencil-and-paper  mode. 

Pilot  testing  of  the  automated  survey  system  has  been  completed,  and  the  software  and 
procedures  have  been  refined.  The  principal  test  of  the  automated  survey  administration 
procedure  and  the,  experimental  scales  will  take  place  in  a  controlled  laboratory  environment  at 
the  Learning  Abilities  Measurement  Program  (LAMP)  facility  at  Lackland  AFB  during  August 
and  September  1993,  during  which  time  the  scales  will  be  administered  twice  (at  2-week 
intervals)  to  a  sizable  sample  of  high-  and  low-aptitude  airmen  in  a  wide  variety  of  technical  and 
nontechnical  specialties.  Built-in  validation  procedures  will  include:  (a)  asking  the  respondent 
to  indicate  which  tasks  in  the  job  description  he  or  she  did  on  the  day  before  taking  the  survey 
and  comparing  these  responses  to  the  frequency  and  time  spent  data  on  the  rated  tasks  across  all 
respondents  (there  should  be  a  fairly  high  correlation  if  the  survey  procedure  is  valid),  and  (b) 
asking  the  respondent  to  select  the  time  spent  value  which  he/she  considers  to  be  most  accurate 
when  there  is  a  large  discrepancy  between  the  criterion  and  the  experimental  scale  values  (the 
source  of  each  value  will  not  be  revealed,  so  as  not  to  bias  the  responses;  the  chosen  value 
should  most  frequently  be  the  one  from  the  criterion  scale,  if  it  is  truly  a  valid  criterion). 

The  expected  payoffs  of  the  automated  survey  technologies  are  many: 

1.  Elimination  of  printing,  mailing,  return,  and  data  entry  costs  of  over  $1,000,000  per  year. 

2.  More  rapid  field  surveying  of  respondents  (reducing  the  process  from  7-9  to  1-2  months). 
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3.  Potentially  more  valid  and  reliable  data  (due  to  tailored  presentation  and  feedback  loops). 

4.  Potentially  more  effective  and  efficient  occupational  analysis. 

5.  The  ability  to  do  spot  surveys  on  an  as-needed  basis  or  annual  resurveys  at  a  stated  time, 
rather  than  blanket  surveys  every  2  to  8  years. 

6.  Potentially  invaluable  tools  for  conducting  management  engineering,  productive  capacity, 
and  performance  measurement  studies  (which  require  measures  of  real  time). 

The  Tailored  Task  List  software  is  designed  to  guide  the  respondent  through  lengthy  task  lists 
in  the  most  efficient  manner,  so  that  the  respondent  will  not  encounter  tasks  that  have  a  low 
probability  of  being  performed  by  him  or  her,  given  his  or  her  prior  task  responses.  The 
Tailored  Task  List  procedure  can  also  be  used  in  conjunction  with  the  computer-administered 
survey  system  to  rapidly  develop  job  descriptions  and  associated  real-time  estimates  for  tasks  in 
order  to  establish  organizational  manpower  requirements.  A  potentially  high-payoff  opportunity 
for  this  PC-based  software  is  its  ability  to  rapidly  develop  (at  the  hands  of  local  supervisors  or 
OJT  trainers)  task  training  lists  tailored  to  the  task-level  training  requirements  of  an  organization 
and  its  individual  workers.  It  may  similarly  be  used  at  Utilization  and  Training  (U&T) 
conferences  by  functional  managers  to  help  develop  task  training  lists  tailored  to  meet  Air  Force 
specialty  (AFS),  shredout,  weapon  system,  or  other  general  or  specific  task-level  training 
requirements. 

The  Automated  Survey/Tailored  Task  List  system  project  has  a  very  significant  potential  to 
positively  impact  the  occupational  analysis  programs  of  all  the  military  services,  through  better 
scaling  and  computer-based  surveying.  By  using  the  electronic  transfer  of  inventories  and 
completed  case  data  files,  it  should  also  encourage  similar  electronic  transfer  of  completed 
reports  and  analysis  files.  This  process  is  already  underway,  to  a  considerable  degree,  in  the 
work  USAFOMS  is  doing  in  support  of  the  Base  Training  System  (BTS).  Here  specially  sorted 
job  descriptions  are  used  as  input  files  from  which  the  BTS  develops  generic  position 
descriptions  as  well  as  the  master  task  lists  for  each  specialty. 

Multilevel  Analysis  of  Task  Data 

There  are  extensive  amounts  of  task-level  information  now  available  on  most  military 
occupations,  which  are  used  effectively  for  a  variety  of  purposes.  Organizing  such  task 
information  into  task  modules,  jobs,  and  higher-order  categories  allows  the  data  to  be  applied 
to  more  global  issues  and  problems,  and  the  summarized  data  can  be  used  to  develop  more 
realistic  models  or  simulations  of  occupational  structures  and  requirements  (Perrin  et  al.,  1988; 
Vaughan  et  al.,  1989).  Existing  data  already  permit  comprehensive  organizational  modeling. 
Some  present  analyses  involve  multiple  specialties,  multiple  categories  of  personnel  (enlisted, 
officer,  civilian),  or  even  multiple  services  (interservice  or  joint-service  projects).  Given  the 
substantial  value  of  task-based  information  and  analyses,  multilevel  studies  focused  on  task 
modules  and  other  higher-order  groupings  have  considerable  potential  for  applications  in 
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modeling  military  organizations  to  assist  military  decision-makers  in  evaluating  proposed 
organizational  restructuring,  interventions,  and/or  other  organizational  changes  (Mitchell, 
Phalen,  &  Hand,  1992). 

Current  experimental  work  is  focusing  on  adjusting  the  task  clustering  algorithm  or  expanding 
the  task  co-performance  similarity  matrix  to  yield  more  interpretable  groupings  of  tasks,  so  as 
to  distinguish  meaningful  subgroups  among  the  large  numbers  of  commonly  performed  tasks. 
We  need  to  be  able  to  develop  multilevel  taxonomies  of  variables  which  can  be  linked,  either 
directly  or  indirectly,  with  tasks  or  groups  of  tasks  (TMs)  and  with  knowledges,  skills,  and 
abilities  (KSAs)-potentially  at  many  different  levels. 

These  linkages  could  be  estimated  directly,  by  having  SMEs  map  the  KSAs  required  for  every 
task.  This  is  a  formidable  undertaking  in  terms  of  the  sheer  volume  of  tasks  and  potential 
KSAs.  It  can  and  is  being  done  in  some  circumstances  (see,  for  example,  Weissmuller,  Dittmar, 
&  Moon,  1993).  The  same  type  of  linkage  can  be  achieved  indirectly  by  having  job  incumbents 
rate  the  tasks  they  perform  in  their  jobs,  and  subsequently  identify  and  rate  the  KSAs  they  use 
in  their  jobs.  The  tasks  and  KSAs  can  be  clustered  separately,  and  linkages  between  the  task 
modules  (TMs)  and  the  KSA  modules  (KMs)  can  be  established  by  determining  the  commonality 
of  cases  involved  in  both  the  TM  and  KSA  clustering  solutions,  as  reflected  by  their  CODAP 
CORCAS  indexes.  Other  measures  might  include  evidence  generated  directly  from  paired 
comparisons  of  task  ratings  with  KSA  ratings  across  all  cases  in  the  sample.  For  example,  if 
a  task  rated  high  on  time  spent  (salient  task)  is  associated  with  a  zero-rated  KSA  by  a  significant 
number  of  cases,  it  can  be  assumed  that  this  task  and  KSA  are  not  linked;  whereas,  if  there  is 
a  significantly  high  correlation  between  a  task  and  a  KSA  when  both  have  non-zero  ratings,  one 
can  assume  that  there  is  a  linkage  between  them. 

Another  indirect  linkage  technology  being  developed  involves  the  use  of  graph  theory  to  establish 
degrees  of  "connectedness"  (e.g.,  similarity  or  dorninance)  between  items  for  which  only  a  small 
number  of  direct  comparisons  are  available.  For  example,  if  task  A  is  estimated  by  one  rater 
to  be  similar  for  weapon  systems  X  and  Y,  and  is  also  estimated  to  be  similar  for  weapon 
systems  Y  and  Z  by  a  second  rater  and  for  systems  V  and  Z  by  a  third  rater,  we  can  say  that 
X  and  Z  are  one-connected  through  Y,  and  that  X  and  V  are  two-connected  through  Y  and  Z. 
In  this  manner,  it  is  quite  possible  to  build  nonredundant  connectedness  networks  which  yield 
several  times  as  many  indirect  linkages  as  the  direct  linkages  furnished  by  the  raters. 

This  technology  will  make  it  possible  to  perform  complex  occupational  analyses  requiring  large 
numbers  of  paired  comparisons,  yet  requiring  only  a  limited  number  of  raters  who  have 
compared  relatively  small  subsets  of  items  (e.g.,  four  to  six  weapon  systems)  per  task. 

Semantic  Analysis 

Another  emergent  technology  falls  under  the  heading  of  Semantic-Assisted  Analysis  Technology 
(SAAT).  SAAT  actually  covers  a  set  of  analysis  principles  which  have  been  converted  into 
unique  computer  code  for  a  variety  of  specific  applications  since  its  initial  conception  in  1984. 
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The  purpose  of  SAAT  is  to  provide  a  linkage  table  (mapping)  from  items  in  one  data  set  into 
matching  items  in  another  data  set.  This  may  be  mapping  tasks  from  one  job  inventory  into 
another  (such  as  Time  1  -  Time  2  analysis  or  merging  AFSs),  mapping  tasks  into  testing  areas 
for  promotion  tests  (Dittmar,  Weissmuller,  Haynes,  &  Phalen,  1989),  mapping  tasks  into 
Maintenance  Data  Collection  (MDC)  records  of  work  performed,  or  mapping  tasks  into  Logistic 
Support  Analysis  Records  (LSARs).  Once  the  linkage  table  has  been  established,  the  task-level 
data  can  be  summarized  and  used  for  the  purpose  at  hand. 

Currently,  SAAT  operates  on  well-structured  lists  of  items,  such  as  task  statements  or 
descriptions  of  individual  data  items.  Because  of  the  clear  syntax  used  in  these  lists,  determining 
parts  of  speech  for  the  noun  and  verb  phrases  is  straightforward.  The  SAAT  involves  a  noun 
and  verb  similarity  assessment,  with  procedures  for  compensating  for  abbreviations,  acronyms, 
misspelled  words,  plural  and  other  word  forms,  and  synonyms  (if  desired).  Without  further 
linguistic  development,  semantic-assisted  clustering  can  be  tailored  to  improve  the  definition  of 
task  co-performance  modules  by  measuring  the  strength  of  semantic  linkage  of  the  component 
task  statements  comprising  a  task  module  with  another  task  module,  even  though  general 
performance  levels  differ  (e.g.,  subset  vs.  superset  modules). 

Additional  work  is  planned  to  expand  the  linkage  methodology  to  link  task  statements  into 
paragraphs,  sections,  and  chapters  of  textual  training  materials  (such  as  Career  Development 
Courses  and  technical  manuals);  such  expansion  would  require  an  enhanced  linguistic  analysis 
process  to  properly  parse  and  relate  sentence  fragments.  This  process  will  also  require  using 
additional  dictionaries  and  thesauruses  in  which  words  are  labeled  as  parts  of  speech  (adjective, 
adverb,  etc.)  and  synonymous  terms  are  identified,  and  in  which  rules  of  grammar  operate. 
Given  this  expanded  capability,  it  would  be  possible  to  use  logic  rules  (such  as  the  law  of 
transitivity)  to  establish  relations  between  tasks  or  task  modules  and  weapon  systems,  other 
equipment,  or  procedures,  which  themselves  are  linked  with  required  knowledges,  and 
probabilistic  measures  (based  on  frequency  of  co-occurrence  of  words  and  word  combinations 
in  a  well-defined  context).  An  extended  matrix  of  relationships  between  TMs  or  jobs  versus 
weapon  systems  or  other  equipment  operated  or  maintained  can  be  developed,  which  would  be 
extremely  useful  in  determining  job  requirements  (KSAs,  educational  background,  training,  etc.) 
for  optimal  person-job  matching.  The  key  to  optimizing  such  person-job  matches  (and  thus  to 
improved  productivity)  is  the  realistic  determination  of  real  job  requirements  (Ward,  Vaughan, 
Mitchell,  Driskill,  &  Ruck,  1992). 


Future  Requirements 

Although  much  has  been  accomplished  in  recent  years,  and  much  is  planned  in  the  near  future, 
much  still  remains  to  be  done  to  improve  and  refine  military  occupational  analysis  technologies. 
We  need  future  basic  research  into  the  nature  of  tasks  and  the  construction  of  meaningful  task 
modules  and  higher-level  aggregations  of  work.  We  will  need  your  help  in  developing  a  realistic 
statement  of  the  requirement  for  this  work,  as  well  as  in  finding  appropriate  funding.  We  also 
need  further  applied  research  to  further  enhance  the  automated  analysis  programs  and  bring  them 
more  into  the  realm  of  state-of-the-art  Artificial  Intelligence.  Again,  your  help  is  needed  in 
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defining,  justifying,  and  funding  this  advanced  development  work.  Finally,  we  also  need  your 
support  and  money  to  enhance  CODAP  software  and  make  it  more  user-friendly  and  interactive 
on  RISC  and  PC  computer  systems.  Given  recent  advances  in  such  hardware  systems,  we 
should  be  aiming  for  an  analysis  system  where  each  occupational  analyst  can  run  CODAP 
interactively  on  his  or  her  desk-top  computer,  thus  ending  our  reliance  on  expensive  mainframe 
systems  and  batch  operations. 
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m.  THE  TEAM  APPROACH  TO  OCCUPATIONAL  ANALYSIS 
IN  THE  CANADIAN  FORCES 

Captain  L.  M.  Phillippo 
G,  J.  Higgs 

The  Canadian  Forces  (CF)  are  a  fully  integrated  service  with  a  single  military  occupational 
structure  (MOS).  Occupational  analysis  (OA)  is  the  principal  means  applied  to  evaluate  the 
appropriateness  of  the  MOS  for  meeting  current  and  future  operational  requirements,  its  value 
being  that  it  can  best  determine  or  validate  the  job  performance  requirements  on  which  the 
structure  is  based.  In  order  to  evaluate  and  develop  occupational  structures,  however,  the  OA 
effort  shifts  from  simply  producing  Job  descriptions  to  the  more  complex  process  of  packaging 
those  job  requirements  in  a  rational  way  to  meet  operational,  organizational,  and  personnel 
management  requirements. 

Occupational  structure  dictates  the  employment  patterns  within  an  occupation,  and  the  career 
paths  for  individuals.  The  content  of  the  occupational  specifications  produced  from  job  data 
will,  in  turn,  drive  such  things  as  training.  In  the  CF,  the  occupational  analyst  enters  the  world 
of  occupational  structure,  or  military  classification,  and  therefore  must  become  involved  in  the 
structure  development  process  and  must  have  access  to  information  that  is  not  typically 
accessible  through  the  job  data  base  alone.  Operational  doctrine,  equipment  maintenance 
doctrine,  and  future  perspectives  with  a  view  to  the  changing  nature  of  jobs  and  force  structure 
through  downsizing  are  but  a  few  of  the  variables  which  can  have  a  significant  impact  on  the 
eventual  jobs  to  be  performed  and  the  occupational  structure  or  personnel  management 
framework  that  is  placed  around  these  jobs.  The  CF  has  developed  a  highly  interactive  and 
successful  multimethod  OA  process  to  meet  its  needs.  A  key  ingredient  in  this  process  is  the 
use  of  a  full-time  OA  project  team  to  carry  out  all  phases  of  work  in  an  OA  project.  Prior  to 
a  description  of  the  composition  and  operations  of  this  team,  the  rationale  for  its  employ  should 
be  discussed. 


Multimethod  Process 

As  an  applied  science,  OA  is  used  to  identify  the  job  performance  requirements  from  which  a 
large  number  of  human  resource  activities  flow.  The  job  analysis  methods  available  are 
bewildering  in  both  number  and  variety.  Though  there  is  scant  research  on  the  subject,  the  few 
comparisons  made  of  different  OA  methods  indicate  that  there  is  no  single  method  that  can  meet 
all  of  the  needs  of  every  client  (see,  for  example.  Ash,  1988).  As  each  method  yields  different 
data  products,  it  seems  reasonable  to  expect  that  each  would  be  differentially  suitable  across 
applications.  Figure  4  illustrates  the  major  direct  uses  made  of  OA  data  in  the  CF,  with  each 
application  routinely  accomplished  by  OA  teams  denoted  by  an  asterisk. 

Although  Task  Inventory  TI/CODAP  has  been  at  the  heart  of  the  CF  OA  methodology  for  over 
the  past  20  years,  it  has  been  necessary  to  venture  into  new  ways  of  using  the  very  powerful 
CODAP  software  for  the  handling  of  other  than  task  taxonomies,  such  as  knowledge  and  skill 
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items.  In  its  aim  to  find  the  ideal  method,  the  CF  has  developed  an  approach  that  embraces 
features  from  several  different  practiced  OA  methods  and  which  guides  a  dedicated  project  team 
to  fulfilling  the  wide-ranging  project  objectives  and  structure  mandate  previously  mentioned. 
With  a  view  to  gaining  maximum  benefit  from  an  OA  project,  senior  personnel  managers  and 
commanders  have  generally  endorsed  the  approach  because  it  remains  objective  but  ensures,  and 
demands,  their  involvement  throughout  the  life  of  the  project. 


*  Occupational  structuring  (classification) 

*  Occupational  specification  writing 

*  Job  description  writing 

Manpower  planning 

*  Employment  and  career  progression  planning 

*  Training 

Organizational  planning 

Recruiting  and  selection 

- 

Performance  evaluation 

Compensation  and  benefits 

Occupational  data  retrieval 

(Note:  *  denotes  duties  fulfilled  by  OA  teams) 

Figure  4.  Applications  of  OA  data  in  the  Canadian  Forces. 


The  OA  Team 

From  the  list  of  data  applications  in  Figure  4  one  can  deduce  the  need  for  a  level  of  expertise 
not  typically  found  in  any  one  occupational  analyst.  One  can  also  deduce  that  some  of  these 
applications  cannot  be  finalized  by  the  analyst  alone.  The  job  expert,  or  occupational 
representative,  is  essential  to  their  accomplishment.  These  facts  dictate,  to  a  large  extent,  the 
need  for  a  full-time  OA  project  team  to  carry  out  the  work. 

Team  Composition 

A  typical  OA  project  team  is  comprised  of  a  team  leader,  staff  analyst(s),  and  subject-matter 
expert(s). 

Tpam  Ti>adpr  The  team  leader  is  an  experienced  analyst  who  has  been  an  understudy  for 
at  least  one  complete  OA  project  and  is  the  team’s  working  supervisor,  who  assigns  and 


monitors  the  duties  and  tasks  of  team  members,  and  who  generally  oversees  or  directs  all  aspects 
of  team  operations  and  administration  in  meeting  project  milestones  and  objectives.  As  is  the 
case  with  all  CF  military  officer  analysts,  the  team  leader  is  at  the  captain  rank  level  and  will 
be  from  either  the  Personnel  Selection  Officer  (PSEL)  or  the  Training  Development  Officer 
(TRGDEV)  military  occupational  code  (MOC).  These  selected  officers  will  typically  spend  3 
to  4  years  in  the  OA  specialty  area.  Given  the  broad  and  comprehensive  mandate  of  CF  OA 
projects,  the  training  and  development  of  team  leaders  is  an  investment  that  suffers  from  military 
rotation.  To  maintain  the  necessary  level  of  continuity  and  expertise  on  the  team,  it  is  planned 
to  have  civilian  staff  analysts  assigned  to  OA  teams  in  the  future. 

Staff  Anaivsttsl.  Staff  analysts  consist  of  junior  officers  from  the  PSEL  and  TRGDEV  MOCs 
and  senior-ranked  noncommissioned  members  (enlisted  grade)  from  several  different  MOCs. 
As  mentioned  previously,  these  individuals  fill  permanently  established  billets  for  3  or  4  years; 
from  their  ranks  future  team  leaders  are  selected. 

Subiect-Matter  Exoertts).  Each  MOC  being  studied  is  represented  by  one  or  more  of  its 
members,  who  are  assigned  to  the  Directorate  of  Manpower  Planning  (DMP)  for  the  duration 
of  the  project.  The  number  of  SMEs  assigned  depends  upon  the  scope  of  the  project,  the  size 
of  the  MOC  being  reviewed,  and  the  diversity  of  work  performed.  The  most  important  of  the 
criteria  for  selection  of  an  SME  is  the  individual’s  experience  in,  and  knowledge  of,  the  basic 
jobs  of  the  MOC;  therefore,  the  SME  is  usually  in  the  rank  of  captain/major  or  warrant  officer/ 
master  warrant  officer.  The  SME  is  expected  to  perform,  at  a  basic  level,  all  of  the  tasks  of  a 
new  staff  analyst,  while  continuously  providing  expertise  and  practical  understanding  of  the 
MOC.  Although  the  senior  SME  may  in  some  instances  out-rank  the  team  leader,  rarely  is  there 
conflict,  as  each  has  a  job  to  do  and  must  complement  the  other  throughout  the  project.  The 
team  leader  knows  the  direction  that  must  be  taken  by  the  team,  and  the  SME  provides  the 
expert  advice  that,  overcomes  many  OA  procedural  and  coordination  problems. 

Training 

The  team  is  provided  with  a  series  of  formal  in-house  training  sessions  which  are  phased-in  to 
precede  each  phase  of  the  OA  project.  Through  these  sessions  and  on-the-job  training,  the  team 
as  a  whole  learns  the  concepts  and  procedures  of  the  OA  process  including  performing 
background  research;  interviewing  techniques;  inventory  development;  field  administration  of 
the  OA  questionnaire;  data  analysis;  conducting  a  training  needs  analysis;  developing 
employment  and  career  patterns;  evaluating  or  developing  occupational  structure;  and  production 
of  draft  occupational  specifications. 

The  Team  Concept:  Strengths  and  Weaknesses 


Strengths 

There  are  obviously  many  advantages  in  having  a  full-time  project  team  to  perform  the  wide 
variety  and  large  number  of  team  tasks  during  a  typical  OA  project  in  the  CF.  These  include; 
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1.  Continuity  and  Consensus.  Because  the  team  as  a  whole  participates  in  all  phases  of  an 
OA,  a  high  degree  of  understanding  of  the  problems  in  an  MOC  is  developed;  continuity  of 
thought  usually  prevails  in  the  interpretation  of  the  data  collected;  and  consensus  is  easier  to 
achieve  on  how  to  handle  and  apply  the  analysis  findings  to  correcting  problems.  This  is 
particularly  so  when  dealing  with  aspects  of  occupational  structure  or  when  developing  career 
patterns  for  an  MOC. 

2.  Pooling  of  Resources.  In  the  small  CF  OA  Section,  only  a  project  team  is  capable  of 
accomplishing  the  large  number  of  team  tasks  within  a  reasonably  short  period  of  time. 

3.  Leadership  Opportunity.  For  the  team  leaders,  who  are  at  the  junior  officer  rank  level, 
managing  and  guiding  a  team  through  the  complex  OA  process  is  an  excellent  opportunity  for 
applying  and  displaying  leadership  skills. 

4.  Learning  Opportunity.  For  new  analysts,  the  opportunity  to  learn  and  develop  analytical 
skills  and  to  practice  these  skills  as  members  of  a  team  engaged  in  an  active  project  is  the 
optimum  learning  experience. 

5.  Benefits  of  the  Team  SMEs.  Employing  SMEs  for  the  duration  of  the  project  is  a  small 
investment  on  the  part  of  project  sponsors,  given  the  potential  impact  the  OA  recommendations 
can  have  on  an  occupation.  Their  education  in  the  OA  process  and  the  training  they  receive  in 
analysis  techniques,  combined  with  their  expert  knowledge,  is  invaluable  to  the  team.  Because 
they  share  common  analyst  training,  they  can  acquire  reasonably  quickly  many  of  the  analyst 
skills  and  enhance  the  quality  of  the  team’s  work  in  such  areas  as  (a)  inventory/questionnaire 
development;  (b)  advising  on  jobs  and  incumbents  for  observation  and  interview;  (c)  facilitating 
contact  with  user  commands,  other  participants,  and  the  project  sponsor;  (d)  providing  a 
practical  understanding  of  the  jobs  and  job  relationships;  (e)  interpreting  the  data;  (f) 
understanding  the  process  of  building  occupational  structure  based  on  valid  job  requirements  and 
not  on  personal  whim  or  bias;  (g)  producing  follow-on  analysis  products  such  as  drafting  new 
occupational  specifications;  (h)  acquiring  a  level  of  objectivity  that  results  in  well-reasoned 
thought  behind  project  report  recommendations;  and  (i)  helping  to  communicate,  legitimize,  and 
market  project  results. 

Host  SMEs  become  highly  knowledgeable  about  their  occupation,  and  many  become  very 
capable  analysts.  Their  post-project  employment  frequently  takes  advantage  of  the  benefits  of 
their  team  tenure.  They  also  become  much  more  valuable  to  their  occupational  managers,  a 
factor  which  can  generally  be  of  benefit  to  their  careers. 

Weaknesses 

Some  weaknesses  in  using  the  team  concept  include: 
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1.  Cost.  The  full-time  employment  of  SMEs  is  obviously  costly;  however,  this  cost  can  be 
viewed  as  money  well  spent,  in  that  extremely  accurate  statements  of  job  requirements  are 
determined,  leading  to  MOC  training  economies  along  with  the  other  benefits  mentioned  above. 

2.  Possible  Biases  of  SMEs.  SMEs  may  embark  on  an  OA  with  hidden  personal  agendas 
or  biases  relating  to  how  they  think  an  occupation  should  be  structured.  Such  bias  may  hinder 
their  ability  to  be  objective,  even  in  the  face  of  hard  data.  This  kind  of  situation  has  been  rare 
in  the  experience  of  DMP  and  can  usually  be  mitigated  by  good  personnel  management  by  the 
team  leader  and  through  structured  analyst  training. 

3.  Inexperience  of  SMEs.  There  are  no  "overnight  analysts."  It  is,  of  course,  unreasonable 
to  expect  SMEs  to  be  instantly  capable  of  a  complete  and  immediate  understanding  of  the 
complex  concepts  of  OA.  However,  SMEs  can  be,  and  usually  are,  highly  productive 
throughout  the  project  and  often  progress,  through  natural  personal  aptitude,  just  as  quickly  as 
new  staff  analysts. 

4.  Possible  Job  Dissatisfaction.  The  fact  that  team  tasks  must  sometimes  be  completed 
without  a  detailed  understanding  of  their  need  or  importance  sometirnes  causes  dissatisfaction 
among  SMEs.  This  is  particularly  true  when  the  SMEs  are  fairly  senior  and  the  tasks  are 
somewhat  mundane  or  administrative  in  nature.  An  effort  must  be  consciously  made  early  in 
the  selection  process  and  to  assigned  SMEs  directly  to  advise  them  about  the  nature  of  all  of 
their  prospective  duties  for  a  project. 

On  Balance 

SMEs  are  a  valuable  ingredient  for  the  team  charged  with  the  conduct  of  an  OA  project.  They 
are  virtually  essential  to  the  inventory  development  and  data  interpretation  processes  using  any 
OA  method;  however,  they  are  indispensable  when  dealing  with  the  broader  issues  of 
employment,  training,  and  career  patterns  during  occupational  structure  evaluation.  The  size 
of  the  team  can  be  problematic;  so,  "smaller  is  best,"  provided  most  of  the  basic  MOC  jobs  can 
be  covered  with  the  SMEs’  expertise.  Part-time  SMEs  and  working  groups  can  always  be 
imported  to  fill  any  gaps  in  the  expertise  of  the  team  SME(s)  and  to  validate  job  inventories 
before  field  survey. 


Conclusions 

Commanders  and  personnel  proponents  at  all  levels  are  demanding  an  ever-increasing  amount 
of  job-related  information  to  promote  or  sustain  proactive  personnel  systems.  In  addition,  the 
military  services  are  currently  experiencing  unprecedented  change,  with  new  roles  and  taskings. 
New  ways  must  be  developed  for  improving  upon  OA  processes  using  the  TI/CODAP 
methodology  if  military  occupational  analysis/survey  programs  are  to  keep  pace  with  such 
change.  It  is  the  considered  opinion  of  the  authors  that  the  team  concept  utilizing  resident  SMEs 
is  one  such  improvement  and  is  a  most  efficient  way  of  meeting  broad  OA  mandates  such  as 
those  now  being  experienced  by  the  Canadian  Forces. 
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IV.  A  STUDY  OF  OCCUPATIONAL  ANALYSIS  PRACTITIONERS 


John  Fugill 
Johnny  Weissmuller 
Wayne  Archer 

The  Network  of  Occupational  Analysts  (NOA)  of  Australia  created  and  distributed  a  job 
inventory  for  CODAP  practitioners-people  involved  in  occupational  analysis  (OA)  projects  using 
the  Comprehensive  Occupational  Data  Analysis  Programs  (CODAP)  software  and  methodology. 
Responses  were  solicited  from  practitioners  worldwide  during  the  summer  of  1991.  This  chapter 
provides  a  broad  overview  of  the  results  of  this  study.  Copies  of  the  complete  report  are 
available  from  the  Institute  for  Job  and  Occupational  Analysis  (IJOA),  926  Toepperwein  Road, 
Converse,  TX,  78109. 


The  Survey  Instrument 

The  survey  instrument,  dated  May  1991,  contained  a  Background  Section  and  seven  major  lists. 
The  Background  Section  solicited  name;  business  telephone  number;  business  address;  type  of 
employer  (commerce,  military,  education,  etc.);  employer  class  (militoy,  civilian);  country  of 
employment  (Australia,  Canada,  or  U.  S.);  job  interest;  use  of  abilities;  use  of  training; 
organization  use  of  language  translations,  Braille,  confidentiality  agreements,  and  security 
regulations;  percent  of  data  automation  that  is  keypunch/keydisk  or  optical  scanner  or  reader; 
and  percent  of  projects  that  involve  employer  associations,  managers  in  commercial  firms, 
military  organizations,  non-military  government  departments,  and  unions. 

The  major  lists  included  a  "check  all  that  apply"  instruction  for  items  under  the  following  topics: 
"In  what  areas  do.you  think  more  CODAP  research  is  needed?";  "In  what  areas  would  you  like 
procedural  guides  to  be  developed  or  improved?";  "In  what  application  areas  have  you  been 
involved  during  the  past  3  years?";  "In  what  industries  or  sectors  have  you  been  doing 
occupational  analysis  work  during  the  last  3  years?";  "How  did  you  learn  to  do  CODAP  work?"; 
"What  tasks  do  you  perform  in  your  current  job?"  (then  rate  relative  time  spent  on  a  9-point 
scale);  and  finally,  "What  knowledge  items  do  you  use  in  your  current  job?"  The  "Areas  of 
Research"  list  had  20  items  and  ranged  from  "Abilities  Inventories"  to  "Veracity  of  Survey 
Responses."  The  "Procedural  Guides"  list  had  7  items  and  ranged  from  "Inventory 
Development"  to  "Survey  Analysis."  The  "Application  Areas  of  OA"  list  had  39  items  and 
ranged  from  "Affirmative  Action"  to  "Working  Conditions."  The  "How  I  Learned  CODAP" 
list  had  5  items  ranging  from  "Accredited  University  or  College  Course"  to  "Self-Instruction" 
(-f  "Other").  The  "Task"  list  had  113  tasks  organized  under  12  Duty  Areas.  The  "Knowledge" 
list  had  80  items  ranging  from  "Analysis  of  Variance"  to  "Worker  Characteristics." 

The  Current  Data  Base 

Data  were  received  and  automated  from  90  CODAP  practitioners.  With  one  exception,  survey 
booklets  from  identifiable  agencies  were  assigned  consecutive  Case  Control  Numbers  (CCNs). 
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The  diversity  and  representativeness  of  the  sample  are  reflected  in  the  following  breakdown  by 
booklet  numbers.  Booklets  from  Australia  are  CCNs  0001  through  0027  (and  0090);  booklets 
from  Canada  are  numbered  0028-0036;  booklets  from  the  United  States  Coast  Guard  are 
numbered  0037-0040;  booklets  from  the  United  States  Navy  are  numbered  0041-0050;  booklets 
from  the  USAF  Occupational  Measurement  Squadron  are  numbered  0051-0081;  booklets  from 
the  USAF  Armstrong  Laboratory,  Human  Resources  Directorate  are  numbered  0082-0085;  and 
booklets  from  U.  S.  contractors  are  numbered  0085-0089.  It  is  hoped  that  in  the  future  other 
agencies  may  wish  to  complete  this  survey.  With  the  atCODAP  system  used  in  this  project,  new 
data  can  be  easily  compared  to  and  integrated  into  this  data  base. 

The  Job-Typing  Results 

Table  1  shows  the  11  job  types  included  in  the  Job-Type  Diagram. 


Table  1 

Job  Types  in  Job-Type  Diagram 

- 

Reporting  Cluster  (Grp  21) 

General  Practitioner  (Grp  45) 

Analyst  (Grp  50) 

Report  Writer/Presenter  (Grp  49) 

Report  Presenter  (Grp  32) 

Analysis  Design  Cluster  (Grp  15) 

Analysis  Method  Designer  (Grp  37) 

Software  Designer  (Grp  54) 

Independent  Job  Types 

Supervisor/Manager  (Grp  40) 

Survey  Materials  Writer  (Grp  35) 

Inventory  Editor  (Grp  22) 

Data  Entry  Clerk  (Grp  17) 

Completed  Booklets  Recorder  (Grp  14) 

Figure  5  shows  the  Job-Type  Diagram  for  identified  groups.  Table  2  shows  the  task  co¬ 
performance  modules  that  were  derived  from  this  data  set  using  standard  procedures. 

Because  this  inventory  was  well  designed,  had  a  small  number  of  respondents  (n  =  90),  a  small 
number  of  tasks  (n  =  113)  and  a  diversity  of  jobs  covered  (Data  Entry  through  Executive),  the 
experimental  enhancements  for  task  clustering  did  not  noticeably  improve  upon  the  clusters 
found  using  existing  methods.  Those  enhancements  will  be  discussed  in  future  papers  on  data 
sets  requiring  improved  resolution. 
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Table  2 


Task  Co-Pejformance  Modules:  CODAP  Practitioners 


Module 

No. 

Module 

Title 

Task  Statements 

1 

Software 

Development-A 

Calculate  expected  outcomes  by  hand 

Confer  with  analyst  on  requested  new  software 

Determine  adequacy  of  existing  software  for  proposed 
applications 

Develop  specifications  for  new  programs 

Decide  whether  to  modify  existing  software  or  develop 
new  software 

2 

Software 

Development-B 

Establish  prototype  control  cards  or  control  specifications 
to  exercise  program  options 

Identify  file  types  required  to  accomplish  objectives 

Test  entire  program  for  functions 

Test  modules  for  functions 

3 

Project 

Management-A 

Define  objectives  of  project 

Advise  managers  on  occupational  analysis  procedures 

Estimate  project  costs 

Monitor  project  costs 

4 

Project 

Management“B 

Draft  work  program  or  schedule 

Gather  information  about  client’s  requirements 

Estimate  computing  requirements 

Estimate  keypunching  or  scanning  requirements 

5 

Project 

Management-C 

Define  communication  line  or  protocols 

Chair  project  meeting 

Convene  project  meeting 

Assign  priorities  or  tasks 

Explain  objectives,  policy  or  practices 

Set  standards  or  performance  objectives 

6 

Project 

Management-D 

Maintain  staff  roster 

Supervise  survey  data  analysts 

Supervise  inventory  developers 
Supervise  survey  administrators 

(table  continues) 
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Table  2 


Task  Co-Performance  Modules:  CODAP  Practitioners 


Module  Module 

No,  Title  _ Task  Statements 


7  Project  Define  ownership  of  survey  information 

Management-E  Evaluate  bidders’  proposals 

Monitor  technical  performance  of  contract 
Draft  contract 


8  Survey  Materials  Identify  sample  of  persons  for  interviews 

-Preparation  Conduct  face-to-face  interview  with  subject  specialist 

Write  Background  Information  section 
Develop  preliminary  inventory  from  source  materials 
Write  supplementary  list  (knowledge,  tools,  machines) 
Write  survey  instructions 

Conduct  face-to-face  interview  with  job  incumbents 
Identify  functional  areas 
Organize  schedule  of  interviews 
Translate  interview  notes  into  task  statements 
Write  up  notes  after  interview 


9  Survey  Materials  Arrange  task  statements  or  other  items  into  lists 

-Editing  Edit  Background  Information  section 

Edit  task  inventory 

Edit  supplementary  list  (knowledge,  tools,  machines) 


10  Report  Writing  Assign  titles  to  job-type  groups 

&  Presenting  Identify  job  types 

Select  groups  of  incumbents  for  whom  job  descriptions 
are  required 

Interpret  task  factor  data 
Write  occupational  analysis  report 
Specify  control  values  for  computer  runs 
Write  work  request  for  computer  analyst 
Edit  report  to  agreed  reading  level 
Correct  errors  or  anomalies  in  report 
Answer  client’s  questions  about  report 
Explain  interpretation  of  computer  printouts 
Prepare  oral  presentation 
Present  oral  report 
Rehearse  oral  presentation 


(table  continues) 
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Table  2 


Task  Co-Performance  Modules:  CODAP  Practitioners 


Module 

No. 

Module 

Title 

Task  Statements 

11 

Field 

Surveying-A 

Explain  field  survey  procedures 

Prepare  oral  briefing  for  survey  respondents 
Supervise  booklet  completion  by  expert  panelists 
Supervise  booklet  completion  by  job  incximbents 

12 

Field 

Surveying-B 

Check  accuracy  of  completed  booklets 

Collate  materials  for  mailing  out 

Maintain  records  of  returned  survey  materials 

Record  survey  responses  by  hand 

- 

13 

Survey  Materials 
-Formatting  and 
Data  Entry 

Prepare  artwork  or  layout  of  survey  materials 

Enter  survey  responses  into  word  processor 
or  computer 

Implement  data  entry  checking  systems  and 
procedures 

Type  inventories  or  instructions 
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V.  AN  OPERATIONAJL  TEST  OF  OA  AND  TIDES  DATA 
FOR  MPT  DECISION-MAKING 


Major  Randy  Agee 
Captain  Kara  B.  W,  Martino 
Ted  Lamb 
J.  R.  Knight 

The  Armstrong  Laboratory/Human  Resources  Directorate  (AL/HR)  conducted  an  operational  test 
of  the  Training  Impact  Decision  System  (TIDES;  previously  the  Training  Decisions  System)  in 
conjunction  with  a  contracting  team  led  by  McDonnell  Douglas  Aerospace.  Occupational 
analysis  (OA)  expertise  and  operational  support  were  provided  by  the  Occupational  Measurement 
Squadron  (USAFOMS).  Plans  for  this  test  were  reported  at  the  Seventh  International 
Occupational  Analysts  Workshop  (Mitchell,  Vaughan,  Knight,  Buckenmyer,  &  Bennett,  1991). 
Interim  progress  on  the  project  was  reported  at  the  last  two  Military  Testing  Association 
conferences  (Coccia,  Mitchell,  Knight,  &  Shrum,  1992;  Mitchell,  Buckenmyer,  Huguley,  & 
Knight,  1991). 


Training  Impact  Decision  System 

TIDES  is  a  computer-based  decision  support  system  specifically  designed  to  support  key 
decision-makers  for  a  specialty  at  the  HQ  USAF  and  major  command  (MAJCOM)  levels.  The 
technology  was  developed  to  extend  task-based  job  analysis  to  permit  more  systematic 
consideration  of  personnel  utilization  factors,  training  costs,  resource  requirements,  capacities, 
and  managers’  preferences  in  determining  the  optimal  allocation  of  training  resources  (Mitchell, 
Vaughan  et  al.,  1992;  Vaughan  et  al.,  1989). 

TIDES  helps  the  Air  Force  functional  and  training  communities  balance  a  specialty’s  training 
needs  versus  resources  and  requirements  to  optimize  career  field  management.  The  technology 
provides  analysts  and  decision-makers  with  a  tool  to  systematically  gather,  integrate,  and  analyze 
information  about  jobs,  tasks,  career  field  assignments,  personnel  flows,  and  the  technical 
training  system  within  an  AFS.  By  dynamically  modeling  a  specialty’s  career  flow  patterns, 
TIDES  provides  a  "what-if"  capability  to  assess  the  long-term  impact  of  current  and  future 
constraints  stemming  from  changing  training  and  personnel  policies  and  fiscal  resources. 

The  basic  functions  performed  by  TIDES  are  analyses  to  help  determine  what  tasks  associated 
with  a  specialty  to  train,  when  to  provide  that  training  (i.e. ,  to  first-term  airmen  or  second-term 
airmen),  and  in  what  setting  the  training  should  be  provided.  Currently,  four  training  settings 
are  assessed:  formal  classroom  training  including  airman  basic  residence  courses  and 
supplemental  training;  supervised  hands-on  training  such  as  is  found  in  field  training  detachments 
(FTDs)  and  laboratories;  self-paced  study  (i.e.,  correspondence  courses  or  CDCs);  and  OJT. 

The  analysis  approach  begins  with  an  assessment  of  the  occupational  survey  report  (OSR) 
information  from  USAFOMS.  The  OSR  jobs  are  refined  in  TIDES  by  grouping  individual  task 
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data  into  task  modules  (TMs),  which  are  the  basic  units  of  analyses  in  TIDES.  TMs  are 
developed  through  a  CODAP  clustering  technique;  tasks  are  reorganized  into  initial  groupings 
based  on  co-performance  as  expressed  in  the  occupational  survey  data.  These  initial  groupings 
are  then  refined  by  subject-matter  experts  (SMEs)  on  the  basis  of  shared  skills  and  knowledges, 
and  shared  resources  needed  to  train  the  tasks. 

Use  of  TMs  improves  manageability  by  reducing  the  number  of  comparisons  necessary  from  the 
500  to  2,000  tasks  characteristic  of  a  job  inventory  to  a  smaller  number  of  coherent  groups  of 
tasks.  Use  of  TMs  also  reduces  the  possibility  of  overestimating  training  requirements,  by 
reducing  the  redundancy  associated  with  other  means  of  task  clustering. 

Jobs  are  described  in  terms  of  TMs  as  are  training  courses  and  OJT  requirements.  TMs  are  also 
the  basis  for  surveying  selected  personnel  from  the  specialty  to  provide  data  on  their  individual 
job  and  training  histories.  This  information  helps  analysts  understand  the  flow  of  airmen  from 
one  job  or  training  program  to  another  within  the  specialty.  TIDES  analysts  also  survey 
technical  training  and  FTD  trainers  to  collect  estimates  of  the  time  and  resources  required  to 
train  TMs  in  various  settings.  SMEs  estimate  airman  proficiency  on  TMs  as  a  function  of  time 
spent  in  training  within  the  different  settings. 

The  proficiency  ratings  result  in  the  generation  of  allocation  or  learning  curves.  A  task  module 
allocation  curve  can  be  likened  to  the  law  of  diminishing  returns.  It  is  a  negatively  accelerating 
curve  based  on  a  polynomial  regression  equation  (Proficiency  =  #  classroom  hours  -  # 
classroom  hrs^)  -1-  (#  CDC  hours  -  #  CDC  hrs")  +  (#  FTD  hrs  -  #  FTD  hrs")  +  (#  OJT  hrs  - 
#  OJT  hrs^).  Using  TIDES,  an  analyst  can  determine  the  number  of  training  hours  (beyond 
those  already  expended  in  formal  courses)  needed  to  achieve  100%  proficiency.  For  example, 
in  Figure  6,  if  an  airman  receives  10  hours  of  training  on  a  task  module  in  the  classroom,  and 
8  hours  of  training  on  that  same  module  in  an  FTD,  the  curve  shows  that  at  the  end  of 
schoolhouse  training,  the  airman  is  approximately  30%  proficient  on  that  task  module;  at  the 
completion  of  FTD,  the  airman  is  about  36%  proficient  on  that  task  module.  Because  the  cmwes 
are  additive,  the  airman  is  about  66%  proficient  after  formal  training.  In  order  to  make  up  the 
34%  deficit  through  OJT,  the  allocation  curve  indicates  that  about  7  hours  are  needed  to  raise 
proficiency  to  100%. 

TIDES  integrates  the  information  collected  and  identifies  the  current  utilization  and  training 
pattern  (U&TP)  of  a  specialty.  The  U&TP  is  a  qualitative  and  quantitative  description  of  how 
airmen  move  through  jobs  and  training  throughout  their  careers.  Figure  7  shows  the  types  of 
jobs  available  at  different  points  in  a  career,  as  well  as  the  technical  training  and  professional 
military  education  (PME)  associated  with  these  jobs.  It  also  shows  that  as  airmen  flow  through 
their  careers,  typically  a  variety  of  jobs  are  available  to  them  as  they  increase  skill  level  and 
experience.  As  shown  in  Figure  7,  airmen  enter  Basic  Military  Training  and  then  attend  an 
airman  basic  residence  technical  school  for  their  specialty.  Following  completion  of  technical 
training  courses,  airmen  are  assigned  to  one  of  several  jobs.  Associated  with  that  job  there  may 
be  other  forms  of  technical  training-an  FTD  or  self-paced  study  such  as  a  CDC. 


35 


PROFICIENCY  {%) 


ON'THE-JOS  TRAININQ 


TRAINING  HOURS 


Figure  6.  Example  task  module  allocation  curve. 


Figure  7.  Example  AFS  Utilization  and  Training  Pattern. 


As  in  "the  real  Air  Force,"  there  is  a  probability  associated  with  whether  airmen  will  receive 
the  formal  training  associated  with  the  job  to  which  they  were  assigned.  After  airmen  complete 
their  first  assignment,  they  enter  a  collect  pool  in  the  TIDES  model  where  they  may  be  joined 
by  cross-trainees  from  other  specialties,  exit  the  system  via  separation  or  cross-training,  reenlist 
into  the  same  job,  or  be  assigned  to  a  second  assignment.  Again,  there  are  probabilities 
associated  with  being  assigned  to  a  particular  job,  and  a  further  probability  that  technical  training 
needed  in  that  job  will  be  provided.  By  assessing  the  thousands  of  airmen  within  a  specialty  and 
summing  across  all  individuals  flowing  through  the  model  to  establish  a  steady  state,  we  have 
found  that  this  method  of  computer  simulation  can  provide  realistic  and  stable  estimates  of  real- 
world  requirements.  Thus,  by  simulating  the  flow  of  airmen,  along  with  their  probabilities  of 
receiving  or  not  receiving  training,  the  long-range  impact  on  the  mission,  formal  training,  and 
OJT  can  be  determined. 

Figure  7  can  be  used  to  identify  OJT  requirements  because  any  associated  training  not  provided 
within  the  realm  of  technical  training  must  be  provided  through  OJT.  Determining  OJT  costs 
and  requirements  is  an  area  in  which  the  Air  Force  has  traditionally  not  been  able  to  make  many 
inroads.  By  using  TIDES,  functional  managers  can  determine  the  relative  cost  of  OJT  in  the 
form  of  salary  of  trainers  and  trainees,  number  of  hours  required  to  deliver  training,  and  the  cost 
of  variable,  expendable  resources  used  in  the  training. 

Once  the  current  U&TP  is  established,  it  can  be  used  as  a  baseline  against  which  alternative 
patterns  can  be  evaluated.  Alternative  utilization  and  training  patterns  are  the  means  by  which 
functional  managers  can  perform  what-if  analyses,  and  evaluate  proposed  utilization  changes 
prior  to  implementation,  to  determine  the  most  effective  combination  of  training  factors  and 
enable  airmen  to  achieve  full  proficiency  within  available  resources.  Capacity  analyses  are 
performed  to  balance  training  requirements  in  terms  of  people,  critical  equipment,  and  mission 
against  the  current  and  projected  available  resources.  One  can  also  impose  constraints  on  the 
modeling  program  to  determine  the  amount  of  training  and  the  content  of  training  necessary  to 
achieve  a  specified  goal. 

Using  TIDES,  decision-makers  can  assess  how  changes  in  such  areas  as  budget,  MPT  policies, 
and  job  restructuring  impact  other  factors  such  as  training  costs  and  capacities  in  terms  of 
training  drivers  (i.e.,  number  of  trainers  and  trainees,  availability  of  critical  equipment  or 
resources,  and  relative  costs  associated  with  personnel  and  resources).  By  providing  the 
capability  to  assess  proposed  U&TPs  prior  to  implementation,  functional  managers  can  maximize 
operational  readiness,  training  effectiveness,  and  efficiency  while  minimizing  training  costs,  yet 
still  provide  the  means  to  produce  the  highest  quality,  fully  trained  forces. 

Determination  of  the  optimal  allocation  of  training  resources  is  made  through  accurate  trade-off 
analyses  of  alternative  training  decisions;  this  what-if  modeling  capability  is  the  hallmark  of  the 
TIDES  technology.  Examples  of  the  types  of  analyses  that  can  be  performed  for  an  AFS 
include: 
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-  How  can  OJT  resource  capacity  limitations  be  alleviated? 

-  What  are  the  cost  and  capacity  consequences  of  eliminating  (or  adding) 
a  formal  technical  training  course? 

-  How  will  increased  student  flow  through  the  entry-level  course  affect  training? 

-  What  is  the  long-term  impact  of  a  25%  reduction  in  TDY-to-school  costs  on  a 
specialty’s  mission? 

-  What  is  the  optimal  allocation  of  training  tasks  within  a  formal  training  course? 

-  Can  we  reduce  costs  by  recruiting  industry-trained  people  and  then  sending  them  to 
direct  duty? 

-  Will  shortening  (or  lengthening)  reenlistment  rates  reduce  costs  and  still  provide 
sufficient  AFS  coverage? 

-  How  will  a  change  in  travel  and  per  diem  costs  or  salary  levels  affect  training 
costs? 

TIDES  empowers  top-level  decision-makers  to  determine  optimal  utilization  of  training 
resources.  Such  capabilities  are  essential  if  the  Air  Force  is  to  plan  the  full  use  of  limited 
resources  and  still  achieve  the  readiness  needed  to  implement  Global  Reach/Global  Power. 

The  need  for  TIDES  is  highlighted  by  General  McPeak’s  1992  Year  of  Training  objectives, 
many  of  which  can  be  supported  through  implementation  of  the  TIDES  technology.  Another 
factor  pointing  to  the  need  for  TIDES  is  our  complex  technical  training  system.  There  is  a 
wealth  of  information  pertinent  to  the  resolution  of  MPT  decisions,  but  each  function  within  the 
Air  Force  has  a  small  piece  of  the  puzzle,  and  there  is  no  consolidated  data  base  which 
integrates  the  information  into  a  form  readable  by  the  applicable  users.  TIDES  complements 
other  MPT  technologies  such  as  the  Base  Training  System  (BTS)  in  the  integration  of  data. 

Yet  a  third  reason  underscoring  the  importance  of  TIDES  is  the  massive  reorganization  we  are 
all  experiencing.  As  specialties  and  MAJCOMs  merge,  we  must  make  smarter  training 
decisions.  As  part  of  that  effort  there  is  an  increased  emphasis  on  utilization  and  training 
workshops  (U&TWs)  and  the  role  of  the  functional  manager.  With  the  elimination  of  Training 
Staff  Officers  in  the  Air  Training  Command,  many  of  their  responsibilities  have  shifted  to  the 
functional  managers,  including  the  responsibility  for  determining  technical  course  content. 

Functional  managers  have  also  been  tasked  with  developing  and  implementing  Career  Field 
Education  and  Training  Plans  (CFETPs).  These  plans  outline  the  flow  of  airmen  through  a 
specialty,  including  the  points  at  which  airmen  should  transition  to  new  job  assignments,  receive 
technical  training,  and  receive  PME.  The  plans  also  outline  training  objectives  for  a  specialty 
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and  list  the  training  opportunities  available  to  personnel  in  that  specialty.  In  our  on-going 
research,  we  found  that  this  is  an  area  in  which  functional  managers  may  have  incomplete 
information.  They  may  not  be  aware  of  many  of  the  MAJCOM-provided  and  contractor- 
provided  training  programs.  TIDES  helps  identity  low-visibility  courses. 

Although  functional  managers  have  a  full  plate,  their  resources  are  limited;  and  it  is  anticipated 
that  they  will  have  to  begin  programming  for  training  dollars  in  the  near  future.  TIDES 
provides  functional  managers  with  the  hard  data  and  estimates  they  need  to  bolster  their 
justifications  for  the  funding  needed  to  support  their  training  objectives. 

Operational  Test 

To  test  TIDES’  ability  to  support  functional  managers  and  the  demands  of  a  U&TW,  the  TIDES 
technical  team  (consisting  of  members  from  AL/HR,  USAFOMS,  and  contractors)  has  been 
working  with  the  HQ  USAF  functional  manager  in  the  305X4,  Electronic  and  Computer 
Switching  Systems,  career  field.  This  chapter  presents  an  update  to  the  paper  presented  at  the 
previous  International  Occupational  Analysts  Workshop  (Mitchell,  Vaughan  etal.,  1991),  which 
outlined  the  plan  to  utilize  a  U&TW  as  a  test  bed  for  assessing  the  effectiveness  of  TIDES. 

To  implement  the  test  plan,  the  TIDES  technical  team  met  with  the  305X4  HQ  USAF  functional 
manager  to  determine  his  needs;  after  several  rounds  of  discussions,  he  identified  several 
candidate  scenarios  for  potential  application  to  the  305X4  specialty  and  asked  for  analyses  of 
each  scenario.  The  alternatives  modeled  were 

1.  the  current  U&TP,  with  the  existing  seven  shreds  merging  at  the  5-skill  level; 

2.  implementation  of  a  common  electronics  principles  (EP)  course  and  elimination  of  all 
airman  basic  residence  (ABR)  courses; 

3.  the  merger  of  all  shreds  into  a  single-ladder  career  field; 

4.  extending  the  existing  shreds  to  the  7-skill  level  and  providing  separate  ABRs  for  each; 

5.  reducing  the  number  of  shreds  to  three  or  four;  and 

6.  reducing  the  number  of  shreds  to  three  or  four,  extending  the  shreds  to  the  5-skill  level 
and  merging  them  at  the  7-skill  level. 

To  initiate  the  analyses  for  each  scenario,  USAFOMS  data  had  to  be  assessed,  and  additional 
SME  data  had  to  be  gathered.  The  contractors  worked  closely  with  the  USAFOMS  occupational 
analyst  to  determine  the  job  structure  and  task  clusters.  They  jointly  developed  the  job  typology 
reported  in  the  OSR  for  the  305X4  specialty;  this  cooperation  enhanced  the  overall  analysis 
process. 
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To  collect  SME  information,  the  R&D  team  hosted  a  preliminary  TIDES  data  collection 
workshop.  The  functional  manager  selected  the  MAJCOM  and  training  representatives  and 
requested  their  participation  in  the  4-day  meeting.  With  the  functional  manager  in  attendance, 
the  contractors  briefed  TIDES  and  administered  the  TIDES  surveys  to  the  participants. 

The  senior  305X4  NCOs  who  assembled  for  the  May  1992  workshop  validated,  titled,  and 
refined  the  task  modules  originally  created  using  the  co-performance  algorithm.  The  SMEs  also 
provided  judgments  of  time  and  resource  requirements  and  determined  proficiency  ratings  for 
each  TM  in  each  of  the  training  settings.  The  final  results  grouped  the  original  657  occupational 
inventory  tasks  into  73  modules  which  can  be  characterized  as  follows: 

largest  module  -  36  tasks 

smallest  module  -  1  task  (two  of  these;  three  modules  contained  2  tasks) 
most  modules  contained  between  5  and  20  tasks 
five  obsolete  modules 

each  module  had  a  specific  title  reflecting  the  commonality  of  tasks 

A  majority  of  the  11  data  collection  workshop  participants  later  attended  the  U&TW.  This 
overlap  greatly  facilitated  decision-making  at  the  U&TW,  because  many  of  the  attendants  were 
already  familiar  with  and  had  confidence  in  the  data.  Data  and  results  of  the  analyses  were 
presented  at  the  formal  305X4  U&TW  conducted  in  March  1993. 

The  first  effort  involved  using  TMs  to  determine  commonality  among  the  seven  3-skill-level¬ 
awarding  technical  schools  (ABRs).  Using  the  Plan  of  Instruction  (POI)  for  each  of  the  seven 
courses  and  the  match  that  USAFOMS  prepared  (each  inventory  task  was  matched  to  a  POI 
block  of  instruction),  participants  were  able  to  approximate  the  number  of  hours  devoted  to  each 
module  in  each  course.  The  module  grouping  made  a  more  comprehensive  common  denomina¬ 
tor  across  all  the  functional  areas  of  this  AES  than  the  Specialty  Training  Standard  (STS). 

Rather  than  the  traditional  STS  scrubdown  that  a  U&TW  would  perform,  the  group  chose  to  go 
through  each  module  task  by  task  using  the  expertise  represented  and  determine  the  proper 
training  setting  for  each  of  the  tasks  in  the  73  modules.  In  this  case,  the  training  settings  were 
defined  as  technical  school/laboratory,  self-paced,  or  OJT.  Table  3  depicts  several  example 
modules  as  they  were  applied  to  the  present  courses  in  terms  of  hours  spent  in  the  technical 
training  environment. 

Using  the  TMs  to  compare  across  present  courses  and  looking  at  present  POIs,  the  group  was 
quickly  able  to  visualize  the  similarities  and  differences.  Using  their  extensive  knowledge  of  the 
career  field’s  changing  technology  and  present  technical  instruction,  a  new  proposed  training 
structure  (initial  skills,  upgrade  OJT,  and  advanced  courses)  was  built  using  the  task  modules. 
Examples  of  the  modules  are  shown  in  Table  4  with  the  proposed  number  of  hours  to  be  devoted 
to  training  in  the  new  technical  training  environment 
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Table  3 


Present  Courses  -  APS  305X4 


E SHRED 

L  SHRED 

R  SHRED 

T SHRED 

M  SHRED 

2 

PERFORM  GENERAL  MAINTENANCE 

13.5 

60 

51 

54 

47.5 

3 

MAINTAIN  DISC  SYSTEMS 

6 

26 

41 

0 

32 

4 

MAINTAIN  PRINTERS 

0 

22 

12 

28.5 

25 

5 

MAINTAIN  DISPLAY  EQUIPMENT 

25.5 

11.5 

7 

33 

178 

6 

MAINTAIN  MAGNETIC  TAPE  UNITS 

10 

0 

21.5 

25.5 

16 

7 

CONSOLE  AND  OPERATOR  PANEL  MAINT 

0 

25 

16 

3 

22 

8 

PROCESSOR  &  MEMORY  ASSEMBLIES  MAINT 

46.5 

54 

77.5 

26.5 

209 

9 

POWER  SUPPLY  MAINTENANCE 

7 

27.5 

13.5 

0 

0 

10 

BENCH  CHECK  ASSEMBLIES 

0 

9.5 

7.5 

23 

0 

Table  4 

Proposed  Courses  -  AFS  305X4 

COMMON 

A  SHRED 

B SHRED 

C SHRED 

2 

PERFORM  GENERAL  MAINTENANCE 

18 

119.5 

1 

62 

3 

MAINTAIN  DISC  SYSTEMS 

18 

19.5 

27 

4 

MAINTAIN  PRINTERS 

18 

1 

23 

5 

MAINTAIN  DISPLAY  EQUIIMENT 

38 

64.5 

2 

13 

6 

MAINTAIN  MAGNETIC  TAPE  UNITS 

18 

14 

2 

7 

CONSOLE  AND  OPERATOR  PANEL  MAINT 

22 

6 

26 

8 

PROCESSOR  &  MEMORY  ASSEMBLIES  MAINT 

58 

29.5 

2 

55 

9 

POWER  SUPPLY  MAINTENANCE 

19 

1 

28 

10 

BENCH  CHECK  ASSEMBLIES 

11 

TROUBLESHOOT  TO  COMPONENT  LEVEL 

102 

58.5 

135 

41 


Based  on  TIDES  input,  the  existing  seven  shreds  of  the  305X4  specialty  were  restructured  to  - 
three  shreds  which  merge  at  the  7-skilI  level.  This  streamlining  will  reduce  costs,  improve 
training,  and  enhance  assignment  and  promotion  opportunities  for  the  career  field. 

Throughout  the  workshop,  the  contracting  team  was  available  to  perform  near-real-time  analyses. 
For  example,  when  the  U&TW  participants  needed  additional  shred  data  to  augment  their 
specialty  restructuring  task,  the  contractors  presented  the  information  the  following  morning. 

Also,  when  the  functional  manager  requested  data  on  the  effects  of  increasing  the  average  time 
on  station  to  48  months,  the  next  morning  the  contractors  reported  their  analyses  which  indicated 
a  substantial  reduction  in  required  OJT. 


Evaluation 

At  the  completion  of  the  U&TW,  the  TIDES  team  solicited  feedback  on  the  utility  of  TIDES  to 
the  decision-making  process.  Participants  were  asked  to  rate  such  items  as  how  accurately  the 
current  U&TP  reflected  the  specialty;  how  much  the  data  from  the  alternative  U&TPs  aided 
specialty  restructuring;  how  much  TIDES  analyses  contributed  to  training  allocation;  and  how 
much  TIDES  data  helped  in  estimating  OJT  requirements.  Participants  were  also  asked  to 
suggest  additional  types  of  information  that  might  have  been  helpful. 

The  AFS  305X4  workshop  was  held  at  an  R&D  base  and  the  logistics  of  the  meeting  were 
handled  by  AL/HR  and  USAFOMS  to  provide  ready  access  to  OSR  and  TIDES  data  bases.  This 
location  proved  very  useful  in  that  participants  were  isolated  from  normal  operational  pressures. 
Future  U&TWs  may  be  much  more  fruitful  if  they  are  held  at  off-site  locations,  with  logistics 
and  arrangements  handled  by  a  hosting  organization. 

The  biggest  lesson  learned  involved  the  need  for  complete  data  collection.  Once  an  SME  body 
is  assembled,  it  is  imperative  that  all  needed  data  be  obtained  by  that  time.  It  is  very  difficult 
to  complete  data  files  after  a  group  has  returned  to  their  normal  operational  duties  and  locations 
and  must  be  contacted  by  phone  or  mail. 

All  of  the  22  MAJCOM  and  training  participants  found  the  TIDES  data  to  be  useful  and  accurate 
in  the  representation  of  the  305X4  specialty,  and  all  but  one  wholeheartedly  recommended 
including  TIDES  analyses  for  other  specialties  in  future  U&TWs.  Participants  stated  that  the 
what-if  analyses  greatly  facilitated  their  difficult  task  of  redefining  the  structure  of  the  specialty. 
Respondents  further  reported  that  TIDES  data  provided  an  excellent  means  for  conveying 
complex  information.  Eighty-two  percent  (82%)  reported  that  the  TIDES  data  were  excellent  or 
above  average  in  complementing  the  USAFOMS  data  traditionally  presented  at  U&TWs. 
Ninety-five  percent  (95%)  reported  an  enhanced  ability  to  determine  skill-awarding-course 
content,  and  eighty-six  percent  (86%)  reported  an  improved  ability  to  allocate  training.  Eighty- 
two  percent  (82%)  stated  that  TIDES  data  greatly  helped  to  estimate  OJT  requirements,  and 
seventy-three  percent  (73%)  cited  an  improved  means  to  determine  resource  requirements. 

As  a  result  of  the  highly  successful  integration  of  USAFOMS  data  and  TIDES  data,  the 
functional  manager  is  interested  in  pursuing  similar  support  for  the  other  specialties  he  manages. 
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In  a  20  April  1993  letter  to  HQ  ATC/TT  on  "Use  of  the  Training  Impact  Decision  System 
(TIDES),"  Brigadier  General  William  M.  Douglass,  Director  of  Maintenance,  HQ  USAF/LGM, 
noted,  "This  is  the  first  time  TIDES  was  used  as  part  of  a  U&TW  and  the  success  of  the  U&TW 
is  due  largely  to  its  use. "  He  went  on  to  say,  "We  are  excited  about  this  technology  and  look 
forward  to  expanding  its  use  to  additional  maintenance  specialties. " 

AL/HR  and  the  contractors  are  providing  continued  support  to  the  functional  manager.  Post- 
U&TW  activities  will  consist  of  additional  modeling  of  newly  defined  scenarios.  Additional  data 
collection  and  analysis  will  continue,  as  another  meeting  of  this  group  is  tentatively  scheduled 
for  late  fall  of  this  year.  One  analysis  was  completed  which  had  to  do  with  the  functional 
manager’s  concern  over  the  possible  loss  of  FTD  training. 

Three  scenarios  were  constructed  based  on  this  concern: 

1.  Total  loss  of  follow-on  training  after  technical  school  completion-no  5-level  or  7-level  or 
equipment-enhancement  courses.  (The  only  courses  available  would  be  the  basic  ABR  and  four 
PME  programs.) 

2.  Loss  of  Field  Training  Detachment  support.  (This  means  all  the  generic  FTD  courses, 
such  as  Technical  Orders  and  soldering,  and,  more  importantly,  the  specific  FTD  305X4  courses 
taught  at  Tinker  and  Offutt  AFBs.) 

3.  Loss  of  all  technical  school  follow-on  and  enroute  5/7-level  training.  (FTD  training  would 
continue  as  is.) 

[Note:  PME  and  the  basic  ABR  would  remain  the  same  in  all  scenarios.] 

All  of  the  above  scenarios  have  since  been  modeled  and  results  made  available  to  the  functional 
manager. 

The  newly  proposed  C  shred  is  actually  a  merger  of  the  old  L  (SACDIN)  shred  and  the  Missile 
Communications  Specialty  (AFS  362X3).  To  aid  in  this  endeavor,  the  contractor  task  clustered 
past  occupational  survey  data.  These  data  are  presently  being  matched  to  362X3  POI  blocks 
necessary  for  inclusion  in  the  newly  proposed  C-shred  course.  Of  the  362X3  task  modules,  it 
appears  that  only  six  or  so  need  to  be  included  in  the  newly  identified  career  field.  Work  is 
continuing  in  refining  these  courses. 


Summary 

The  operational  test  was  highly  successful.  TIDES  can  be  used  to  dynamically  model  the  long¬ 
term  personnel  and  training  issue  impacts  on  an  AFS  for  various  combinations  of  training 
content,  sites,  and  proposed  policy  changes  within  the  training  system.  TIDES  is  a  powerful 
methodology  to  aid  key  decision-makers  at  the  MAJCOM  and  USAF  levels  in  determining  the 
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optimal  utilization  of  resources  necessary  to  support  the  mission  of  an  AFS.  Some  of  the 
expected  benefits  of  TIDES  are  as  follows: 

-  Career  field  planning/management  support  tool. 

-  Training  cost  forecasting  model,  including  OJT  cost  estimates. 

-  Simulation  and  evaluation  of  current  and  alternate  AFS  utilization  and  training  patterns. 

-  Data  support  for  U&TWs. 

-  Integration  of  comprehensive  MPT  information  into  a  single  data  base  for  each  AFS. 

-  MPT  decision  impact  assessment  capability  in  terms  of  costs,  resource  requirements,  and 
training  system  capacities. 

TIDES  has  been  involved  in  several  previous  U&TWs,  including  the  Security  Police  (811XX), 
B-1  Avionics  Test  Station  (AFS  451X7),  and  Aerospace  Propulsion  (454X0 A/B).  However, 
the  entire  scope  of  the  TIDES  program  was  not  yet  available  to  the  participants  at  the  previous 
workshops  during  the  initial  R&D.  The  AFS  305X4  U&TW  used  the  full  robustness  of  the 
TIDES  programs  to  look  at  all  issues  in  the  MPT  arena  (classification  structure,  training 
equipment  needed,  training  costs,  unit  capabilities,  etc.).  Particularly  revolutionary  was  the  fact 
that  task  modules  were  used  rather  than  the  standard  STS.  Task  modules  simplified  the  complex 
requirements  of  this  diverse  specialty  and  helped  participants  deal  directly  with  relevant 
behavioral  objectives.  Senior  managers  were  able  to  view  their  career  field  from  different 
perspectives  when  the  many  varying  "career  path"  scenarios  were  modeled.  Furthermore,  task 
modules  were  used  to  construct  general  content  for  the  technical  ABR  courses. 
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VI.  MILITARY  OCCUPATIONAL  ANALYSIS;  FUTURE  DIRECTIONS 

Winston  Bennett,  Jr. 

David  S.  Vaughan 

Hendrick  W.  Ruck 

Occupational  or  job  analysis  provides  a  detailed  specification  of  the  requirements  for 
performance  within  jobs  (Cascio,  1991).  The  previous  chapters  have  served  to  highlight  uses 
of  occupational  analysis  information  within  the  military  context.  In  addition,  job  or  occupational 
analysis  data  are  used  for  personnel  recruitment,  selection,  placement,  training  and  development, 
and  performance  appraisal  (Cascio,  1991;  Muchinsky,  1990).  Typically,  job  analysis  data  are 
used  to  portray  the  current  state  of  tasks  within  jobs  in  an  organization  for  these  purposes. 

We  believe  that  there  are  key  areas  of  opportunity  for  future  research  in  occupational  analysis 
and  application.  These  include  improvements  to  existing  OA  practice  and  extending  OA 
applications  for  multilevel  analysis  and  dynamic  organizational  modeling.  Each  of  these  areas 
will  be  discussed  in  the  following  sections. 

Enproviiig  Ciirreiit  OA  Practice 

Currently,  the  practice  of  OA-conducting  OA  studies  and  applying  OA  results-is  a  very  costly, 
labor-intensive  activity.  A  great  deal  of  time  and  effort  is  required,  for  example,  to  build  a  task 
inventory,  administer  and  process  the  various  surveys  based  on  the  inventory,  analyze  the  data, 
and  use  the  results  for  MPT  planning  and  decision-making.  The  time  required  for  job 
incumbents  and  SMEs  to  complete  surveys,  in  the  aggregate,  is  quite  large.  Also,  a  great  deal 
of  time  and  effort  is  required  to  train  occupational  analysts  and  support  staff.  One  significant 
future  direction  for  OA  lies  in  improving  current  practice  so  as  to  obtain  better  and  more  widely 
used  results  with  less  cost  and  effort.  Current  OA  practice  can  be  improved  in  several  ways, 
including  (a)  improved  task  inventory  development  processes;  (b)  developing  and  implementing 
advanced  automated  tools  for  OA  analysis;  (c)  developing  automated,  adaptive  survey 
administration;  (d)  improving  rating  response  scaling  methods;  and  (e)  developing  software  user 
interfaces. 

Improyed  Task  Inventory  Development 

The  starting  point  for  task-based  OA  is  the  task  inventory.  The  overall  value  of  an  OA  study 
is  critically  dependent  on  the  quality  of  its  task  list.  However,  task  list  development  is  probably 
the  least  systematically  studied  aspect  of  existing  OA  technology.  We  believe  that  a  major  focus 
for  OA-related  R&D  should  be  task  inventory  development.  We  also  believe  that  this  work 
should  consider  the  particular  issues  to  be  addressed  using  the  inventory.  There  has  been 
significant  progress  in  this  area.  For  example,  Driskill,  Gorman,  Weissmuller,  Tucker,  and 
Hand  (1992)  have  developed  methods  for  constructing  task  inventories  from  logistics-related  data 
sources  for  aircraft  maintenance  occupations.  This  approach  ensures  that  the  task  inventory 
reflects  major  aircraft  subsystem  differences  and  is  consistent  with  the  way  in  which  the  target 
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occupations  view  their  work,  while  reducing  the  effort  and  subjective  judgment  required  to  build 
a  task  inventory.  We  believe  that  this  kind  of  R&D  should  be  continued  and  extended  to  a 
broader  occupation  range,  and  be  implemented  as  appropriate. 

Advanced  Automated  Tools  for  Occupational  Analysis 

Even  with  adequate  computer  support  for  the  creation  of  statistical  results  and  summaries  of  OA 
data  (e.g.,  CODAP),  the  analysis  process  is  very  labor-intensive.  A  major  portion  of  the 
analysis  effort  is  devoted  to  interpretation  of  statistical  cluster  analysis  for  job  typing  and  task 
clustering  (task  module  development).  Also,  a  great  deal  of  training  is  required  for  one  to  be 
a  successful  analyst.  One  step  that  has  been  taken  to  reduce  both  the  labor  required  for  analysis 
and  the  training  required  is  to  standardize  the  analysis  process.  A  standard  set  of  computer 
products  are  produced,  and  interpreted  and  reported  in  a  standardized  way.  A  disadvantage  of 
this  approach  is  that  specific,  unique  questions  and  issues  in  a  particular  occupation  may  not  be 
addressed  by  the  standard  analysis.  Therefore,  the  users’  needs  may  not  be  effectively  met. 

Advanced  automated  analysis  tools  could  reduce  the  overall  effort  required  for  analysis  and  aid 
in  tailoring  analyses  to  better  meet  users’  needs.  Substantive  work  is  underway  to  use  automated 
tools  for  job  typing  and  task  clustering  (see  Chapter  II).  To  date,  this  work  has  been  quite 
successful.  However,  more  progress  can  and  should  be  made  in  automated  job  typing  and  task 
clustering.  For  example,  tools  should  be  created  for  addressing  specific  user  questions  and 
issues.  These  tools  could  be  used  to  interactively  present  a  wide  variety  of  examples  of 
questions  and  analysis  products  and  procedures  to  address  specific  types  of  questions.  Further, 
these  tools  could  be  integrated  into  an  interactive  computer  program  (perhaps  a  "front  end"  to 
CODAP)  that  gathers  information  from  an  analyst  or  a  user  concerning  questions  and  issues  to 
be  addressed  and  then  provides  advice  concerning  what  analyses  to  conduct  and  what  reports  to 
request. 

Similarly,  artificial  intelligence  (AI)  researchers  have  made  great  strides  in  the  development  of 
rule-based  computer  algorithms  that  capture  expert  performance  on  tasks  and  related  analysis 
activities.  These  algorithms  might  be  used  to  develop  programs  to  accomplish  many  of  the 
activities  now  done  by  analysts,  thereby  reducing  the  time  and  costs  zissociated  with  inventory 
development  and  routine  data  analysis.  However,  these  AI  approaches  have  not  been  applied 
to  the  task-based  occupational  analysis  approach.  Further,  initial  studies  have  been  conducted 
to  demonstrate  the  utility  of  computer-based  inventory  data  collection  programs  (see  Chapter  II). 
However,  research  to  determine  the  effectiveness  of  these  programs  for  large-scale  data 
collection  needs  to  be  conducted.  This  research  would  be  concerned  not  only  with  the  efficiency 
and  cost-effectiveness  of  the  programs,  but  also  with  the  impact  of  the  programs  upon  the  quality 
of  the  data  (e.g.,  reliability  and  validity). 

Automated  Adaptive  Survey  Administratiou 

As  suggested  above,  survey  administration  and  processing  require  a  great  deal  of  time  and 
energy,  from  both  the  occupational  analyst  (to  create  and  administer  surveys  and  to  enter  and 
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process  the  data)  and  the  respondents  (to  complete  the  surveys).  Today,  desk-top  computers  are 
widespread  in  the  Air  Force.  Furthermore,  current  Air  Force  projects  such  as  the  Base  Training 
System  (BTS;  O’Connor,  1990)  are  introducing  the  use  of  computers  at  the  work  site,  with 
supervisors,  for  MPT  management.  The  availability  of  computers  facilitates  the  administration 
and  processing  of  OA  surveys  by  computer,  rather  than  by  paper-and-pencil.  In  addition,  such 
computer-based  survey  administration  would  avoid  the  need  for  paper  reproduction  of  surveys, 
mailing  and  handling,  and  entering  data  into  the  computer.  Such  surveys  could  also  be  adaptive 
(i.e.,  presenting  a  respondent  with  subsets  of  the  overall  task  inventory  appropriate  for  that 
particular  respondent).  This  would  reduce  the  respondents’  workload,  which  could  both  save 
time  and  improve  data  quality.  Presently,  work  is  underway  to  create  automated  OA  surveys 
(see  Chapter  II).  Results  from  this  work  should  be  implemented  as  appropriate. 


Response  Scaling 


Significant  improvement  in  the  reliability  and  validity  of  OA  rating  data  is  expected  to  result 
from  research  being  conducted  by  Phalen  (see  Chapter  II).  This  work  should  result  in  more 
realistic  estimates  of  real  time  spent  in  Air  Force  work  activities.  These  estimates  will  be 
directly  useful  for  manpower  determinations  and  management  engineering  studies. 

Under  current  OA  practice,  task  time  spent  ratings  are  gathered  from  respondents  on  a  relative 
rating  scale.  These  ratings  are  linearly  rescaled  for  analysis.  A  great  deal  of  research  has  been 
done  on  psychophysical  scaling  (scaling  subjective  perceptions  of  objective  physical 
characteristics,  such  as  length,  weight,  and  sound  intensity).  A  general  finding  of  such 
psychophysical  scaling  research  has  been  that  subjective  perceived  magnitude  is  logarithmically 
related  to  associated  physical  scale  values.  These  results  suggest  that  subjective  time  spent 
ratings  may  not  be  linearly  related  to  actual  time  spent.  These  results  also  suggest  that  other 
approaches  to  gathering  subjective  time  spent  ratings  may  produce  more  accurate  estimates  of 
actual  time  spent  than  does  the  relative  rating  scale  currently  used.  We  believe  that  the  issue 
of  how  best  to  gather  time  spent  ratings  and  estimate  actual  times  from  these  data  deserves 
research  attention.  In  fact,  such  research  is  on-going  (see  Chapter  II).  Such  work  should  be 
continued  and  implemented  as  appropriate. 

Current  OA  practice  involves  gathering,  in  addition  to  time  spent  data,  subjective  ratings  of  task 
factors  from  SMEs  (e.g.,  task  learning  difficulty,  recommended  training  emphasis).  Though 
many  of  these  task  factors  are  less  clearly  related  to  underlying  physical  variables,  we  believe 
that  response  scaling  issues  should  be  examined  for  these  task  factor  ratings.  Psychophysical 
scaling  approaches  might  be  useful  for  some  task  factors  (e.g.,  strength  and  stamina,  hazard 
potential).  Another  approach  that  might  improve  both  task  factor  and  time  spent  scaling  involves 
better  establishment  of  the  context  within  which  incumbents/SMEs  provide  ratings.  Basic 
research  related  to  the  nature  of  time  spent  ratings  and  the  rating  context  necessary  to  ensure 
validity  should  be  conducted. 
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Software  User  Interfaces 


The  CODAP  software  system  (Christal  &  Weissmuller,  1988)  has  proven  to  be  an  extremely 
powerful  tool  for  OA.  Research  and  application  activities  have  continued  to  add  analysis 
features  and  capabilities,  such  as  automated  aids  for  job  typing  and  task  clustering,  as  well  as 
to  rehost  the  software  to  operate  on  computer  systems  available  to  the  military  OA  users. 
However,  the  user  interface  for  CODAP  is  a  batch-processing-oriented  design  that  requires 
significant  training  and  experience,  both  with  CODAP  and  with  the  host  computer  operating 
system.  Given  the  batch-mainframe  origins  of  CODAP,  this  is  very  understandable.  However, 
today’s  computer  hardware  and  software  technologies  offer  the  possibility  of  a  much  more  user- 
friendly,  interactive  user  interface.  Such  an  interface  should  be  developed  for  CODAP.  This 
interface  would  significantly  reduce  the  training  and  effort  required  to  use  CODAP.  Also,  the 
interface  could  assist  in  creating  computer  products  to  address  specific  users’  questions  and 
issues.  CODAP  is  currently  being  rehosted  to  operate  on  IBM  RS6000  computers  running  a 
variant  of  the  UNIX  operating  system.  This  rehosting  will  greatly  facilitate  creation  of  an 
interactive,  user-friendly  interface,  because  many  tools  for  such  interfaces  are  available  under 
the  UNIX  operating  system  such  as  the  XWINDOWS  and  MOTIF  graphical  user  interface 
software  and  tools  for  building  such  interfaces.  Research  exploring  Ae  costs  and  benefits  of 
several  interface  approaches  should  be  conducted. 

New  Occupational  Analysis  Applications 

Occupational  analysis  data  are  potentially  useful  for  many  human  resources/MPT  management 
purposes.  Historically,  OA  data  have  been  used  in  the  Department  of  Defense  for  classification 
structure  determination  and  training  planning  purposes  (see  Chapter  V).  In  the  future,  OA 
results  should  be  extended  for  application  in  other  important  MPT  areas.  These  areas  of 
potential  application  include  organizational  right-sizing,  organizational  change  evaluation, 
establishing  physical  and  aptitude  requirements  for  occupations,  determining  assignments, 
establishing  and  evaluating  personnel  policies,  and  reducing  work-related  accidents. 

Occupational  analysis  methods  have  been  explored  in  such  areas  as  the  development  and 
validation  of  a  hazard  potential  task  factor  (Thompson  &  Ruck,  1978).  This  hazard  potential 
task  factor  could  be  useful  in  identifying  tasks  for  attention  (e.g.,  training  or  job  redesign)  for 
reduction  of  accidents.  Similarly,  Weeks  (1984)  and  Gott  (1981)  have  developed  task-based 
methods  for  determining  aptitude  requirements  and  physical  task  performance  requirements, 
respectively. 

Potentially  valuable  new  applications  of  OA  data  exist  in  the  manpower  and  personnel  policy 
areas  as  well.  The  various  services  have  analysis  methods  and  tools  in  these  areas.  For 
example,  the  Air  Force  uses  the  Enlisted  Force  Management  System  (EFMS),  a  system  of 
mathematical  models,  to  support  planning  and  decision-making  relating  to  manpower,  force 
structure,  and  personnel  policy  issues.  For  example,  the  EFMS  is  used  to  predict  the  future  Air 
Force-wide  numbers  of  airmen  at  various  grade  and  years-of-service  levels  and  to  investigate 
various  ways  of  making  that  expected  structure  more  nearly  match  the  objective  structure  (the 
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structure  that  should  exist,  given  Congressional  constraints  on  authorizations  by  grade,  etc.). 
The  EFMS  predicts  a  variety  of  Air  Force-wide  and  occupation-specific  outcome  variables, 
based  on  many  manpower  and  personnel  parameters.  These  input  parameters,  or  predictor 
variables,  are  estimated  from  various  existing  Air  Force  manpower  and  personnel  data  files. 

OA  data  currently  have  no  role  in  the  EFMS  or  in  other  Air  Force  macro-level  MPT  planning 
and  analysis  models,  yet  the  use  of  OA  data  has  the  potential  to  significantly  enhance  such 
models,  both  in  terms  of  accuracy  and  in  terms  of  richness  and  usefulness  of  outputs.  For 
example,  in  the  EFMS,  real  variation  within  jobs  and  occupations,  as  well  as  between 
occupations,  is  ignored  or  assumed  away.  That  is,  the  impacts  of  such  simplifying  assumptions 
on  the  model  outputs  remain  untenable.  Furthermore,  systematic  consideration  of  such 
occupation  and  job  characteristics  would  permit  different  policies  and  procedures  to  be  used, 
which,  in  turn,  would  permit  more  effective  management  of  the  various  occupations  while  better 
meeting  overall  force  constraints. 

We  also  recommend  that  significant  additional  issues  within  the  classification  structure  and 
training  areas  be  addressed  by  OA  technology.  In  classification  structure,  for  example,  current 
OA  data  have  proven  very  useful  for  examining  single  occupations  or  small  sets  of  closely 
related  occupations  (e.g.,  the  two  Air  Force  Security  Police  occupations).  However,  OA  data 
are  less  usefol  for  addressing  broader  occupational  structure/specialty  structure  issues.  Current 
OA  practice  provides  little  assistance  in  determining  the  best  way  to  structure  work  in  a  broad 
area,  such  as  all  Air  Force  aircraft  maintenance  jobs. 

Similarly,  OA  data  have  proven  very  useful  for  training  planning  purposes.  However, 
significant  training  issues  remain  unaddressed  by  OA  technology.  Results  from  Ruck  et  al. 
(1987)  support  prioritizing  tasks  for  training  for  a  specified  target  population  (i.e.,  first-term 
airmen).  However,  those  results  provide  only  indirect  support  for  allocating  task  training  to 
formal  technical  schools,  correspondence  courses,  OJT,  or  other  training  delivery  locations  and 
methods.  The  fundamental  drivers  of  these  training  allocation  decisions  relate  to  overall  training 
costs  and  resource  constraints.  Furthermore,  overall  training  costs  and  resource  constraints  are 
influenced  by  many  non-training  factors  such  as  job  structures,  personnel  assignment  policies, 
geographic  location/dispersion,  organization  manning,  and  the  availability  of  equipment,  among 
others  (Mitchell  et  al.,  1993). 

Conventional  approaches  to  studying  training  impacts  on  organizations  would  relate  training 
directly  to  organizational  productivity.  This  approach  has  been  attempted  in  conventional 
training  utility  analysis  studies  (Cascio,  1989;  Mathieu  &  Leonard,  1987).  However,  relating 
training  to  overall  organizational  productivity  can  be  difficult.  In  noncommercial  settings,  such 
as  the  Air  Force,  it  can  be  difficult  even  to  define  the  theoretical  organizational  productivity 
construct  in  a  way  that  is  applicable  to  entire  occupations  (Bennett,  Ruck,  &  Huffcutt,  1992). 
In  a  recent  organizational  modeling  study  (see  Vaughan  et  al.,  1989),  this  problem  was  avoided 
by  fixing  organizational  productivity  at  a  constant  level  and  then  estimating  training  resource 
requirements,  costs,  and  capacities  required  to  achieve  the  fixed  productivity  for  various  training 
scenarios.  In  this  way,  the  influence  of  training  on  meaningful  organizational  variables  (e.g.. 
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operating  budgets  and  resource  requirements)  is  estimated,  permitting  identification  of  the  least 
expensive  way  of  meeting  training  requirements  that  is  consistent  with  training  resource 
availability  constraints. 


Improved  OA  Technology  for  Applications 

Occupational  analysis,  as  currently  practiced  in  the  military,  typically  involves  obtaining  task- 
level  information  concerning  people  performing  such  tasks  (e.g.,  percent  members  performing 
and  percent  time  spent)  and  concerning  task  factors  (e.g.,  task  learning  difficulty,  recommended 
training  emphasis).  Such  task-level  summary  statistics  are  used  directly  or  are  summarized  into 
other  task-level  indexes  (e.g.,  the  Automated  Training  Index)  or  job/occupation-level  summary 
statistics  (e.g.,  job  difficulty).  In  the  future,  the  use  of  more  sophisticated  approaches  for  using 
OA  data  to  support  MPT  planning  and  decision-making  can  greatly  improve  the  usefulness  of 
OA  data  for  such  purposes.  Moreover,  several  of  the  new  applications  discussed  above  (e.g., 
organizational  right-sizing  and  organizational  change  evaluation)  will  require  much  more 
sophisticated  approaches  and  improved  approaches  to  data  collection  and  analysis. 

A  Common  Framework  for  Analysis 

Another  key  issue  in  occupational  analysis  is  related  to  the  development  of  a  common  framework 
for  analysis.  Specifically,  although  the  purposes  for  using  occupational  analysis  data  may  vary, 
the  methods  and  the  units  of  analysis  should  share  a  common  basis.  This  common  basis  would 
facilitate  the  generalizability  of  findings  and  models  from  one  situation  to  another.  Ideally,  a 
single,  multipurpose  analysis  method  could  be  developed  and  applied  in  a  variety  of  settings  and 
for  a  variety  of  purposes.  Future  research  should  focus  on  the  development  of  common  analysis 
methods  encompassing  multiple  levels  of  analysis. 

Aggregating  Task-Level  Information 

One  area  in  which  improvements  are  required  to  better  support  application  of  OA  results  to  MPT 
planning  and  decision-making  is  in  obtaining  and  relating  OA  results  at  multiple  levels  of 
analysis.  This  is  particularly  important  for  addressing  organizational  right-sizing  and  change 
evaluation,  because  both  individual-  and  organizational-level  variables  are  important  for 
addressing  such  issues. 

Though  detailed  task-level  information  is  critical  for  certain  uses,  such  as  training  development 
or  selecting  topics  for  promotion  tests,  such  data  may  be  too  complex  and  specific  for  other 
purposes  (e.g.,  to  facilitate  management  macro-level  decision-making  or  evaluating  the  possible 
impacts  of  organizational  restructuring).  Organizing  task  information  into  task  modules,  jobs, 
and  higher-order  categories  allows  the  data  to  be  applied  to  more  global  issues  and  problems  and 
can  be  used  to  develop  realistic  models  or  simulations  of  occupational  structures  and 
requirements.  Several  task  characteristics  may  be  used  to  group  tasks  into  more  meaningful 
groups.  These  characteristics  include  grouping  by  equipment  used,  by  function,  and  by  task 
co-performance. 
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Task  clustering  using  co-performance  values  as  a  basis  for  developing  task  modules  (TMs)  has 
been  demonstrated  in  several  recent  applications  (see  Mitchell,  Phalen,  &  Hand,  1991;  Perrin 
et  al.,  1988;  Vaughan  et  al.,  1989).  Task  co-performance  is  defined  as  a  measure  of  the 
similarity  of  pairs  of  task  profiles  across  all  the  people  in  an  occupational  survey  sample.  For 
details  of  the  computation  of  measures  of  task  co-performance,  see  Rue  et  al.  (1992). 

TM-level  data  can  be  used  to  provide  very  concise  descriptions  with  which  to  compare  jobs 
within  a  specialty.  For  example,  jobs  can  be  described  by  TM-level  information  which  is  based 
upon  average  time  spent  across  the  tasks  within  each  module  and  percent  members  performing 
data  which  are  average  values  across  the  tasks  within  each  module.  This  provides  comparable 
statistics  for  the  TMs.  It  is  much  easier  to  compare  such  TM-level  job  descriptions  for  various 
jobs  within  a  specialty  than  it  is  to  analyze  task-level  job  descriptions  (typically  ordered  on 
descending  percent  time  spent  or  percent  members  performing).  Thus,  a  modular  approach 
provides  a  structured,  summarized  set  of  data  which  facilitates  between-  or  within-job 
comparisons. 

From  Static  Descriptions  to  Dynamic  Modeling 

A  second  improvement  is  related  to  modeling  and  prediction.  For  more  complex  decision¬ 
making  situations,  the  process  of  extrapolating  from  an  OA  description  of  the  current  situation 
to  what  might  happen  under  various  alternative  future  events  becomes  problematic.  Systematic 
modeling/prediction  methods  can  provide  effective  solutions  to  these  OA  application  problems. 

The  primary  focus  in  military  OA  research  and  development  has  been  on  obtaining  the  most 
detailed  and  accurate  description  of  the  current  situation  (tasks,  jobs,  etc.)  in  an  occupation  or 
organization.  For  example,  a  great  deal  of  effort  has  been  devoted  to  refining  methods  for 
identifying  current  jobs  (e.g.,  job  typing)  and  task  characteristics  (e.g.,  task  factors,  such  as  task 
difficulty).  However,  uses  of  OA  data  usually  involve  making  predictions  about  other  variables 
in  the  future  (possibly  under  various  alternative  scenarios).  A  good,  accurate  picture  of  the 
current  job  and  task  requirements  is  an  essential  starting  point  for  such  prediction.  Typically, 
when  OA  data  are  used  to  support  MPT  planning,  these  predictions  are  only  informally  and 
implicitly  related  to  the  current  baseline  OA  data.  The  usefulness  of  OA  data  could  be  greatly 
increased  by  developing  explicit  models  for  using  OA  data,  along  with  other  relevant  data,  to 
predict  future  organizational  outcome  variables  of  interest  for  MPT  planning. 

This  is  not  to  say  that  modeling  has  been  excluded  from  OA  R&D.  For  example,  a  great  deal 
of  attention  has  been  given  in  several  OA  methodologies  to  modeling  for  compensation  purposes. 
Here,  the  modeling  typically  involves  predicting  current  salaries  for  job  incumbents  from 
OA-based  data  on  incumbents’  jobs.  Similarly,  modeling  has  played  a  key  role  in  validating  OA 
task  factors  such  as  recommended  training  emphasis  (Ruck  et  al.,  1987),  task  hazard  potential 
(Thompson  &  Ruck,  1978),  and  job  difficulty  (Weeks,  1984).  However,  such  modeling  efforts 
have  focused  on  a  single  criterion  variable  and  have  been  relatively  restricted  in  the  predictor 
variables  considered. 
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Organizational-level  variables  have  typically  not  been  included  as  either  predictors  or  outcome 
variables  (see  Bennett  et  al.,  1992).  Several  new  OA  applications  require  predictions  concerning 
organizational-level  outcome  variables  under  various  alternative  organizational  structure/change 
scenarios.  This  is  true  for  both  organizational  right-sizing  and  organizational  change  evaluation 
applications.  Such  organization-level  prediction  is  also  required  for  many  manpower  and 
personnel  policy  issues. 

In  order  to  provide  analytic  support  for  applications  such  as  these,  as  well  as  to  better  support 
existing  applications  (e.g.,  occupational  structuring  and  training  needs  assessment),  OA 
technology  should  be  extended  to  dynamic  modeling  of  organizations.  As  described  above,  the 
EFMS  is  such  an  organizational-level  estimation  or  prediction  model.  Here,  the  principal  R&D 
issues  concern  how  to  best  incorporate  OA  data  to  refine  the  EFMS  models.  For  other  MPT 
planning  applications,  no  macro-level  models  exist  comparable  to  the  EFMS.  In  these  cases, 
organizational  outcomes  of  various  policy  outcomes  (e.g.,  overall  productivity,  cost,  degree  to 
which  resource  constraints  can  be  met)  must  currently  be  estimated  by  planners  from  the  existing 
baseline  OA  data.  Therefore,  OA-based  models  related  to  these  applications  will  need  to  be 
developed  from  scratch. 

The  Training  Decisions  System:  An  OA  Modeling  Case  Study 

How  might  such  a  model  be  built?  One  approach  is  to  estimate  all  model  parameters 
simultaneously  from  a  single  set  of  data  using  a  latent  variable  structural  equation  modeling 
procedure  (e.g.,  Loehlin,  1987).  However,  this  approach  is  often  not  feasible  for  real-world 
applications. 

A  second  approach  involves  developing  a  macro-level  model  for  using  OA  data  for  MPT 
planning.  Such  a  model  has  recently  been  developed  for  strategic  training  planning.  That 
model,  the  Training  Decisions  System  (TDS;  Mitchell,  Vaughan  et  al.,  1992;  Vaughan  et  al., 
1989),  illustrates  how  OA  might  make  the  step  from  pure  description  to  explicit  prediction  of 
future  organizational  outcomes  under  various  complex  policy  scenarios.  Such  a  model  would 
express  important  macro-level  outcome  variables  (e.g.,  organizational  productivity,  mission 
effectiveness)  as  functions  of  important  empirically  related  variables.  These  include  both 
micro-level  variables  that  are  directly  manipulated  in  organizational  interventions  and  more 
macro-level  variables,  such  as  business  and  economic  conditions.  This  type  of  model  would  be 
very  useful  for  exploring  the  quantitative  relationships  among  events  at  various  levels  of 
abstraction.  The  model  could  be  used  to  determine  the  maximum  impact  that  micro-level 
organizational  interventions  can  reasonably  be  expected  to  have  on  macro-level  outcome 
variables,  relative  to  other  uncontrolled  events. 

A  model  of  this  type  will  permit  users  to  estimate  impacts  on  outcome  variables  of  interventions 
at  lower  levels  of  abstraction  (e.g.,  at  the  individual  level)  while  holding  other  variables  constant 
(Vaughan  &  Yadrick,  1992).  Such  a  model  can  also  be  used  to  study  the  relative  sensitivity  of 
outcome  variables  to  the  various  types  of  input  variables.  This  sort  of  sensitivity  analysis  can 
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be  used  to  determine  which  organizational  interventions,  if  any,  are  likely  to  have  practical  value 
in  light  of  the  many  other  factors  that  influence  outcome  variables. 

The  overall  purpose  of  the  TDS  is  to  support  strategic  MPT  planning  for  specialties 
(occupations)  within  the  Air  Force.  The  TDS  meets  this  objective  by  estimating  important 
organizational  impacts  associated  with  structuring  training  for  an  occupation  in  various  ways, 
so  that  trade-offs  associated  with  different  approaches  to  meeting  task  training  requirements 
(e.g.,  mixes  of  classroom,  laboratory,  and  on-the-job  training)  can  be  studied.  TTie  overall 
objective  in  building  the  TDS  model  was  to  take  into  account  the  key  drivers  of  training  resource 
requirements  and  costs,  including  non-training-related  factors. 

A  TDS  model  for  an  organization  or  occupation  requires  a  great  deal  of  data  about  the  job 
structure,  training  structure,  and  tasks  to  be  modeled.  These  data  come  from  many  different 
sources.  The  sources  of  data  include  existing  data  bases  (e.g.,  the  occupational  survey/CODAP 
job  analysis  data  base),  and  related  manpower,  personnel  and  training  data  bases.  In  addition, 
much  of  the  data  come  from  SMEs’  judgments.  As  the  above  discussion  suggests,  a  TDS  model 
for  an  occupation  is  built  in  pieces.  Various  classes  of  model  parameters  are  estimated 
separately,  from  separate  data  sources.  These  classes  of  parameters  are  then  combined  to  form 
a  complete  TDS  model.  For  example,  in  the  U&TP  simulation,  jobs  and  task  content,  training 
courses  and  task  content,  and  transition  probabilities  must  be  estimated  separately  (Mitchell  et 
al.,  1993).  In  many  cases,  multiple  data  sources  must  be  used  for  estimating  a  particular  class 
of  parameter.  FurAermore,  parameters  may  not  be  obtained  using  strict  statistical  estimation 
procedures.  One  reason  for  this  concerns  issues  in  combining  inconsistent  data  from  multiple 
sources.  Another  stronger  reason  concerns  the  generalizability  and  application  of  model  results. 
Historical  data  often  reflect  the  state  of  an  occupation  several  years  ago.  However,  the  model 
results  are  used  to  describe  the  current  situation  and  extrapolate  to  the  future.  Thus,  it  is  often 
necessary  to  adjust  historical  model  parameter  estimates  to  reflect  recent  and  expected  changes. 

Occupational  Analysis-Based  Modeling 

Several  other  significant  MPT  planning  areas  could  benefit  greatly  from  development  of  complex 
OA-based  organizational  models.  One  example  is  in  the  manpower  area.  The  actual  manpower 
required  to  accomplish  work  activities  (e.g.,  numbers  of  task  performances  or  time  spent 
working  on  tasks)  is  a  function  of  many  variables.  Some  of  these  variables,  such  as  task 
characteristics,  job  structure,  and  task  times,  are  clearly  OA-related.  Others  are  more  closely 
related  to  mission  requirements  under  various  scenarios  and  to  overall  manpower  and  personnel 
policies  and  procedures,  such  as  the  occupational  structure  and  classification  procedures. 
Decision-making  concerning  manpower  issues  such  as  occupational  structuring  and  restructuring 
could  benefit  greatly  from  a  model  that  would  estimate  total  costs  (e.g.,  manpower  requirements) 
from  OA  data  and  other  data  under  various  alternative  scenarios  (e.g.,  hypothetical  occupational 
structures). 

Much  of  the  discussion  thus  far  has  focused  on  complex  organizational  outcome  modeling. 
However,  there  are  many  areas  that  would  benefit  greatly  from  simpler  OA-based  modeling. 
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This  was  demonstrated  by  the  success  of  the  compensation  models  and  the  Ruck  et  al.  (1987) 
training  emphasis  research.  In  training  planning,  for  example,  a  simple  model  that  distinguishes 
between  formal  classroom  and  OJT  for  training  on  a  task  would  be  very  useful.  The  model 
could  be  viewed  as  an  approximation  to  the  TDS,  and  could  be  used  in  the  many  circumstances 
where  the  time  and  expense  of  applying  the  TDS  are  not  warranted.  Similarly,  a  simple  model 
that  identifies  those  jobs  or  tasks  in  different  occupations  which  could  be  placed  in  a  "common" 
occupation  would  be  very  useful. 


A  Unified  Theory  of  Work 


The  discussion  above  covers  a  great  deal  of  ground.  To  a  great  extent,  OA  technology  in  the 
DoD  has  been  developed  in  response  to  specific  applied  problems  and  issues  in  MPT  planning 
and  management.  This  is  also  true  of  the  future  directions  for  OA  proposed  above;  that  is,  to 
a  great  extent  they  are  motivated  by  specific  applied  problems  in  MPT  and  in  OA  practice.  As 
a  result,  the  various  directions  are  not  well  connected  or  integrated.  In  the  long  run,  the  DoD 
needs  a  truly  integrated  MPT  planning  and  management  system-an  "ultimate  person-job  match 
system"  (Ward  et  al.,  1992).  Progress  toward  this  ultimate  person-job  match  system,  as  well 
as  synergism  and  focus  in  progress  in  the  future  directions  proposed  above,  requires  an 
integrating  framework.  We  believe  that  a  unified  theory  of  work  is  needed  to  provide  this 
framework  and  to  guide  and  focus  OA  and  related  R&D. 

What  should  such  a  unified  theory  of  work  be  about?  Fundamentally,  such  a  theory  should 
relate  task  performance  to  the  task  and  worker  variables  that  cause  such  performance.  For  tasks, 
these  will  include  the  variables  inherent  in  such  tasks  (e.g.,  task  learning  difficulty,  underlying 
skills  and  knowledges  required  to  perform),  as  well  as  environmental  and  organizational 
structure  and  climate  variables.  For  workers,  these  will  include  variables  inherent  in  people, 
such  as  aptitudes,  abilities,  personality,  and  previous  experience,  as  well  as  variables  related  to 
a  particular  organizational  environment,  such  as  training.  Some  variables,  such  as  motivation, 
will  have  task,  worker,  and  organizational  aspects. 

This  unified  theory  of  work  will  connect  theories  of  human  traits  and  states,  theories  of  task  and 
job  characteristics,  theories  of  job/task  performance,  and  perhaps  theories  of  organizational 
behavior.  For  example,  Mitchell  and  Driskill  (1986)  have  proposed  a  theory  that  relates  training 
to  task  performance,  via  a  series  of  intervening  and  exogenous  variables.  Such  a  theory  could 
be  extended  to  encompass  individual  differences  among  workers  and  tasks,  as  well  as  key 
organizational  and  environmental  variables. 

Developing  and  validating  a  unified  theory  of  work  is  a  long-term,  basic  research  effort; 
however,  we  believe  that  such  a  theory  is  necessary  to  long-term  progress  in  OA.  We  also 
believe  that  a  number  of  key  components  for  such  a  theory  exist.  For  these  reasons,  we  believe 
that  the  time  is  right  to  focus  on  developing  such  a  unified  theory  of  work. 
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Summary 


The  purpose  of  this  chapter  has  been  to  highlight  potential  future  directions  in  occupational 
analysis.  In  an  era  of  declining  research  and  operational  resources,  it  is  likely  that  greater 
demands  will  be  placed  upon  the  OA  process  as  a  means  of  providing  detailed  information  to 
meet  a  multitude  of  organizational  needs.  These  needs  will  include  not  only  "traditional"  uses 
of  occupational  data,  such  as  identifying  job  requirements  and  portraying  the  current  status  of 
military  occupations,  but  also  more  sophisticated  uses  such  as  modeling  alternative  organizational 
structures,  developing  prescriptions  of  future  training  needs,  and  quantifying  the  costs  and 
resource  requirements  for  changes  to  jobs  and  occupations. 

The  research  directions  and  applications  proposed  and  discussed  here  will  provide  the  necessary 
foundation  upon  which  new  uses  of  occupational  analysis  methods  and  products  can  be  based. 
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Vn.  COMMENTS  OF  DISCUSSANTS 


This  chapter  contains  comments  made  by  three  of  the  Symposium’s  four  discussants:  J.  W. 
Cunningham,  North  Carolina  State  University;  Walter  E.  Driskill,  Metrica,  Inc.;  and  James  E. 
Sage,  The  Ohio  State  University.  Written  comments  of  the  other  discussant,  Walter  C.  Borman, 
University  of  South  Florida,  were  not  available  at  the  time  of  publication  of  this  report. 


Discussion:  A  Nomothetic  Approach  to  Job  Analysis 

J.  W.  Cunningham 
North  Carolina  State  University 


In  all  honesty,  I  can  say  I’m  very  impressed  with  the  work  that  has  been  described  here.  As 
Jimmy  Mitchell  already  mentioned  (in  Chapter  I),  the  military  services  have  been  among  the 
largest  contributors  and  supporters  over  the  years  in  advancing  systematic  occupational  analysis. 
And,  of  course,  the  Task  Inventory/CODAP  method  has  played  the  major  role  in  that 
development.  This  method  has  proven  to  be  a  very  powerful  tool  and,  it’s  safe  to  say,  is  the 
most  widely  used  of  all  quantitative  job  analysis  methods.  I  think  the  introduction  of  co¬ 
performance  task  modules,  as  described  earlier,  will  add  considerably  to  its  power. 


The  efforts  reported  by  Phillippo  and  Higgs  (in  Chapter  III)  and  by  Fugill,  Weissmuller,  and 
Archer  (in  Chapter  IV)  have  directed  much-needed  attention  to  the  activities  and  training  of 
people  involved  in  occupational  analysis.  And  the  other  presenters  (authors)  have  described 
innovative  methods  and  procedures  that  will  unquestionably  extend  the  applications  of 
occupational  analysis,  in  both  the  military  and  the  civilian  settings. 

Unfortunately,  there  isn’t  time  to  credit  or  comment  in  detail  on  all  these  important 
contributions.  Instead,  I’ll  take  the  opportunity  to  talk  briefly  about  an  approach  that  I  think 
could  serve  as  a  useful  supplement  to  the  job-task  inventory.  I’ve  labeled  it  the  "nomothetic" 
approach  to  job  analysis.  Its  most  notable  representative  is  the  Position  Analysis  Questionnaire 
(PAQ)  developed  by  Ernest  McCormick  and  his  associates  at  Purdue  University  under  support 
of  the  Office  of  Naval  Research  (McCormick  et  al.,  1972).  A  more  recent  example  is  the 
General  Work  Inventory  (GWI)  that  Rodger  Ballentine,  William  Wimpee,  and  I  have  developed 
and  researched  in  the  Air  Force  setting  (Cunningham  et  al.,  1990).  See  Figure  8  for  details  on 
these  two  approaches. 

The  nomothetic  approach  uses  descriptors  that  are  applicable  to  broad  ranges  of  jobs,  in 
comparison  to  what  might  be  called  the  "ideographic"  approach,  which  uses  descriptors  specific 
to  individual  jobs  or  restricted  job  categories.  Some  abbreviated  examples  of  nomothetic 
descriptors  are  shown  in  Figure  8.  In  this  connection,  McCormick  has  contrasted  worker- 
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oriented  variables,  which  represent  basic  human  behaviors  and  requirements,  with  job-oriented 
variables,  which  represent  technological  content  relevant  to  the  purposes  and  outcomes  of  work. 
For  example,  "Depth  Perception"  (in  the  PAQ  item  examples)  could  anchor  the  worker-oriented 
end  of  McCormick’s  continuum,  whereas  "Electrical/Electronic  Information"  (in  the  GWI 
examples)  is  clearly  job-oriented.  Other  nomothetic  descriptors  fall  in  between  these  two 
extremes,  and  McCormick’s  distinction  is  not  always  so  clear-cut.  The  PAQ  was  designed  to 
contain  mostly  worker-oriented  items,  whereas  the  GWI  contains  about  equal  numbers  of  both 
worker-  and  job-oriented  items. 


NOMOTHETIC  DESCRIPTORS:  General  descriptors  applicable  across  a  broad  spectrum  of  jobs  and  occupations. 


POSITION  ANALYSIS  QUESTIONNAIRE  (PAQ) 
(McCormick,  Jeanneret,  &  Mecham) 


Abbreviated  Item  Examples 


Written  Materials 

Verbal  Sources 

Estimating  Size 

Analyzing  Information  or  Data 

Stationary  Machines/Equipment 

Advising 


Quantitative  Materials 
Depth  Perception 
Inspecting 

Precision  Tools/Instniments 

Assembling/Disassembling 

Persuading 


Visual  Displays 

Estimating  Speed  of  Moving  Objects 

Transcribing 

Measuring  Devices 

Hand-Arm  Steadiness 

Interviewing 


GENERAL  WORK  INVENTORY  (GWI) 
(Cunningham,  Wimpee,  &  Ballentine) 


Abbreviated  Item  Examples 

Ordinary  Speaking 
Basic  Arithmetic 
Coordination  and  Balance 
Maintaining/Repairing/Setting-Up  Machines 
Teaching 


IDEOGRAPHIC  DESCRIPTORS:  Specific  descriptors  applicable  to  individual  jobs  or  job  categories. 


Night  Vision 

Electrical/Electronic  Information 
Visualizing  Objects 
Small  Handtools 
Supervising 

Health  Treating/Caring 


Tables/Graphs/Charts 
Legal/Contractual  Information 
Social  Judgment 

Using  Office  Machinery/Equipment 
Cooperating 


JOB-TASK  INVENTORY 
(Military  Services) 


CRITICAL  INCIDENTS  TECHNIQUE 
(Flanagan) 


Figure  8.  Nomothetic  and  ideographic  descriptors. 
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Examples  of  more  ideographic  kinds  of  descriptors  can  be  found  in  task-inventory  items  and  in 
critical-incident  statements.  In  addition  to  being  much  more  focused  or  fine-grained  than 
nomothetic  descriptors,  the  ideographic  descriptors  are  almost  entirely  job-oriented—the  task- 
inventory  items  probably  moreso  than  the  critical-incident  descriptions. 

As  I  see  it,  the  nomothetic  and  ideographic  descriptors  should  be  considered  complementary  in 
their  uses  and  limitations.  The  nomothetic  variables  allow  for  general  description,  comparison, 
and  classification  of  jobs  across  a  broad  spectrum,  whereas  the  ideographic  descriptors  provide 
more  specific  and  detailed  job  information  that  would  slip  through  a  nomothetic  net.  So,  I 
propose  a  marriage  between  these  two  approaches. 

Potential  uses  of  nomothetic  descriptors  are  shown  in  Figure  9. 


POTENTIAL  USES  OF  NOMOTHETIC  DESCRIPTORS 

NUMERICAL  OCCUPATIONAL  CLASSIFICATION 
TASK  MODULE  CLUSTERING 

ESTIMATING  JOBS’  HUMAN  CAPABILITY  REQUIREMENTS 


Figure  9.  Potential  uses  of  nomothetic  descriptors. 


One  potential  use  of  nomothetic  descriptors  would  be  in  deriving  broad  numerical  occupational 
classification  structures.  For  example,  we’ve  recently  had  encouraging  success  in  hierarchically 
clustering  Air  Force  enlisted  specialties  based  on  their  GWI  ratings. 

Nomothetic  descriptors  also  might  prove  useful  in  rating  and  clustering  co-performance  task 
modules,  producing  higher-order  or  meta-modules.  Such  meta-modules  could  be  interpreted  as 
broad  generic  tasks  which,  if  translated  into  descriptors,  could  be  used  to  profile  and  compare 
relatively  wide  ranges  of  jobs  and  occupations.  Moreover,  it  seems  to  me  that  the  meta¬ 
modules,  along  with  their  constituent  co-performance  modules,  would  provide  a  reasonable  basis 
for  defining  generalizable  knowledges  and  skills. 

Another  application  of  nomothetic  descriptors  would  be  in  estimating  jobs’  general  human 
capability  requirements,  following  McCormick’s  job  component  approach.  The  general  approach 
involves:  first,  defining  a  universal  set  of  "job  components,"  or  nomothetic  descriptors,  and 
then  deriving  requirement  weights  for  those  components  on  various  human  capabilities. 
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Subsequently,  capability-requirement  estimates  can  be  derived  for  any  job  that  has  been  rated 
on  the  weighted  components.  Table  5  depicts  the  requirement  weights  for  q  job  components  on 
k  defined  capabilities,  many  of  which  would  be  represented  by  tests.  There  are  at  least  three 
ways  of  deriving  these  weights,  and  we’re  currently  exploring  them  with  the  GWI. 


Table  5 

Requirement  Weights  for  q  Job  Components  on  k  Defined  Capabilities 


Human  Capabilities 


Components  1  2  .....  Jk_ 


1 

Wll 

W12 

W13 

Wlk 

2 

W21 

W22 

W23 

W2k 

3 

W3r 

W32 

W33 

W3k 

q  Wql 


Wqk 


Figure  10  provides  a  listing  of  the  kinds  of  capabilities  I  think  could  be  accounted  for  by  a 
comprehensive  set  of  nomothetic  descriptors.  These  include  Basic  Physical  and  Psychomotor 
Abilities,  Basic  Mental  Abilities,  General  Nontechnological  Knowledges  and  Skills  (such  as  those 
defined  by  the  SCANS  Commission  and  by  Employment  and  Immigration  Canada),  and 
Generalizable  Technological  Knowledges  and  Skills  (such  as  the  General  Vocational  Capabilities 
posited  some  years  ago  by  Altman  and  Gagne).  Nomothetic  descriptors  could  also  be  readily 
linked  to  work-related  interests. 
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HUMAN  CAPABILITIES  LINKABLE  TO  NOMOTHETIC  JOB  DESCRIPTORS 


SENSORY  CAPACITIES 
(visual,  auditory,  tactual,  etc.) 

BASIC  PHYSICAL  AND  PSYCHOMOTOR  ABIUTIES 
(e.g.,  as  defined  by  Fleishman  &  Quaintance,  1984) 

BASIC  MENTAL  (COGNITIVE)  ABILITIES 
(as  measured  by  standardized  aptitude  tests) 

GENERAL  NONTECHNOLOGICAL  KNOWLEDGES/SKILLS 
(generalizable,  trainable,  and  free  of  occupational  content) 


Basic  Educational  Skills,  Social  Skills,  Communication  Skills,  Etc. 
Examples 

Fundamental  Skills  (SCANS  Commission) 

Generic  Skills  (Employment  &  Immigration  Canada) 
Cross-Functional  Skills  (U.  S.  Employment  Service) 


GENERAL  TECHNOLOGICAL  KNOWLEDGES/SKILLS 
(generalizable,  trainable,  and  explicitly  occupational/technological  in  content) 


Common  Handtools 
Electrical/Electronic  Components 
Tables/Graphs 
Filing 

Rules  of  Effective  Service 
Computer  Usage 


Content  Examples 
Mechanical  Principles 
Connecting/Fastening  Devices 
First  Aid  Practices 
Office  Routine 

Persuasion  and  Sales  Procedures 


Measuring  Devices 
Drawings/Diagrams 
Letter/Report  Writing 
Contracts 

Vehicular  Operation 


CAPABILITIES  REPRESENTED  BY  TASK  MODULES  AND  META-MODULES 

OCCUPATIONAL  AREA  KNOWLEDGES  AND  SKILLS  (META-MODULES) 
(generalizable  within  occupational  areas  or  clusters) 

OCCUPATION-SPECIFIC  KNOWLEDGES  AND  SKILLS  (TASK  CO-PERFORMANCE  MODULES) 

(applicable  within  occupations) 


Figure  10.  Human  capabilities  linked  to  nomothetic  job  descriptors  and  represented  by  task 
modules  and  meta-modules. 


At  more  focused  capability  levels,  the  meta-modules  might  be  considered  as  representative  of 
generalizable  knowledges  and  skills  within  defined  occupational  areas  or  clusters,  and  the  co¬ 
performance  task  modules  could  be  treated  as  representative  of  more  specific  occupational 
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proficiencies.  The  list  shown  in  Figure  10  might  be  viewed  as  a  hierarchical  structure,  in  which 
the  general  capabilities  toward  the  top  are  assumed  prerequisites  for  the  more  specific  ones 
below  them. 

In  addition  to  describing  jobs  and  occupations,  a  set  of  nomothetic  and  module-based  descriptors, 
along  with  capability-requirement  estimates,  could  provide  a  framework  for  profiling  the 
characteristics,  training  and  work  experiences  of  people,  thus  facilitating  the  person-job  matching 
process.  In  this  regard,  we  might  envision  the  development  of  computerized  job  and 
occupational  information  systems.  As  shown  in  Figure  11,  these  parallel  numerical  systems 
would  support  a  number  of  personnel-related  applications. 


AREAS  OF  POTENTIAL  APPLICATION 

OCCUPATIONAL  CLASSIFICATION 

TRAINING 

TRANSFER 

PLACEMENT 

RECRUITMENT 

CAREER  DEVELOPMENT 

PERFORMANCE  APPRAISAL 

HUMAN  RESOURCE  PLANNING 


Figure  11.  Potential  personnel-related  applications. 


These  potential  applications  would  include,  for  example,  occupational  classification,  training, 
transfer,  personnel  placement,  recruitment,  career  development,  performance  appraisal,  and 
human  resource  planning. 
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Discussion:  Some  Critical  Needs  in  Military  Occupational  Analysis 

Walter  E.  Driskill 
Metrica,  Inc. 


After  listening  to  this  series  of  papers,  I  am  struck  by  several  issues,  but  because  of  time 
limitations,  I  will  address  only  two  of  them. 

First,  because  I  was  originally  a  historian,  I  believe  there  is  much  to  gain  from  understanding 
the  roots  of  occupational  analysis  and  how  the  technology  has  progressed  over  the  past  half 
century.  Certainly,  the  need  now  is  as  great,  and  perhaps  greater,  for  information  about  jobs 
in  order  to  deal  with  practical  personnel  problems.  Now,  there  are  40  or  more  occupational  or 
job  analysis  methodologies  that  can  be  located  in  the  literature.  Although  they  approach  the 
study  of  jobs  from  different  perspectives,  they  provide  information  for  such  personnel  functions 
as  selection  and  classification,  training,  career  development,  structuring  jobs,  etc.  Each  of  these 
methodologies  provides  job  information  using  different  scales  and  at  varying  levels  of  specificity. 
As  job  analysts,  we  should  be  aware  of  the  multiple  methodologies^  and  the  importance  of 
looking  at  jobs  from  a  variety  of  perspectives  if  we  are  to  provide  information  useful  to 
operators,  maintainers,  supervisors,  and  managers.  There  is  no  single  best  method  of  job 
analysis.  The  best  approach,  in  fact,  may  be  a  combination  of  methodologies. 

At  Metrica,  we  have  employed  portions  of  several  methodologies  to  probe  research  issues.  For 
example,  during  the  past  year,  we  used  elements  from  Fleishman’s  Abilities  Requirements, 
elements  from  Ballentine  and  Cunningham’s  General  Work  Inventory,  and  the  Comprehensive 
Occupational  Data  Analysis  Programs  (CODAP)  to  study  human  ability  requirements  of  task- 
level  performance.  At  the  present,  we  are  collecting  critical  incidents  information  from  pilots 
who  have  Desert  Storm  combat  experience.  One  approach  we  will  apply  to  understanding  these 
incidents  is  the  use  of  hierarchical  grouping  analysis  from  CODAP,  with  the  ultimate  goal  of 
identifying  the  dimensions  of  combat  pilot  performance  and  the  identification  of  human  attributes 
that  contribute  to  successful  performance. 

I  would  urge  you  to  extend  your  knowledge  and  application  of  job  analysis  methods.  Though 
I  know  most  of  you  are  committed  to  the  task-based  approach  using  CODAP  as  the  analysis  tool, 
please  recognize  that  CODAP  can  be  a  very  useful  analysis  tool  for  other  job  analysis  methods 
as  well-as  evidenced  by  our  recent  study  of  human  ability  requirements.  Data  collection  to 
support  other  methods  can  be  readily  accomplished  in  conjunction  with  the  collection  of  task 
data. 

Second,  the  remaining  papers  highlight  what  in  my  mind  is  a  serious  deficiency  that  has  existed 
for  the  past  25-plus  years.  Simply  put,  extensive  research  and  development  has  been  directed 
at  expanding  and  improving  the  analytic  capabilities  of  CODAP.  I  acknowledge  that  these 
efforts  have  provided  some  significant  advances.  But,  research  to  improve  our  methods  for 
describing  jobs  in  terms  of  task  requirements  has  been  virtually  nonexistent.  To  my  knowledge, 
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this  work  has  been  limited  to  our  work  pertaining  to  methods  for  developing  weapon-system- 
specific  tasks.  Although  this  work  was  tangential  to  another  research  objective,  it  demonstrated 
that  in  the  maintenance  arena  there  are  data  bases  from  which  tasks  describing  work  on  any 
aircraft  or  missile  can  be  compiled.  Software  for  accessing  these  data  bases  and  for  compiling 
tasks  describing  technical  maintenance  activities  is  available. 

Further,  based  on  our  experience  in  accessing  maintenance  data  bases,  I  am  firmly  convinced 
that  other  data  bases  pertaining  to  non-maintenance  occupations  exist  and  could  be  accessed  to 
develop  tentative  task  statements  for  most  other  occupations.  The  most  important  advantage  of 
developing  such  a  technology  is  the  extent  to  which  communication  with  respondents  and 
decision-makers  could  be  immensely  improved. 

And,  it  is  the  improvement  of  such  communication  that  cries  out  to  be  accomplished.  Whether 
we  like  it  or  not,  as  CODAP  practitioners  we  have  to  recognize  that  task  description  leaves  a 
lot  to  be  desired.  In  our  research  over  the  past  4  or  5  years,  we  have  had  occasion  to 
experience  firsthand  the  problems  of  poor  task  description.  We  have  found,  for  example,  that 
as  many  as  35  percent  of  the  respondents  to  a  job  inventory  misunderstood  some  of  the  tasks  in 
the  list. 

The  need  for  better  task  lists  has  existed  since  the  initiation  of  the  Air  Force  occupational 
analysis  program  on  July  2,  1967.  It  is  a  pity  that  research  in  this  area,  which  would  cost  only 
a  small  fraction  of  that  spent  on  expanding  and  improving  the  analytic  capabilities  of  CODAP, 
has  been  neither  supported  nor  funded.  All  of  us  should  recognize  that  though  an  improved 
analytic  capability  may  assist  us  in  analyzing  poor  data,  the  best  analytic  capability  cannot 
overcome  the  shortcomings  and  dangers  of  misinformation. 
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Discussion:  Automated  Seif-Report  Questionnaires-Highlights  of  a  CODAP  PUot  Study 


James  E.  Sage 
The  Ohio  State  University 


Job  and  occupational  analyses  are  managerial  activities  performed  to  gather,  analyze,  and 
synthesize  data  that  describe  the  inter-  and  intra-relationships  of  jobs  within  a  particular 
organization.  These  data  provide  a  foundation  upon  which  the  organization  plans  and  redefines 
itself.  In  addition,  these  data  provide  the  human  resource  manager  and  human  resource 
developer  with  the  legal  foundations  on  which  to  make  employee-related  decisions.  That  is,  job 
and  occupational  analyses  are  analytical  processes  that  investigate  the  relationships  between  the 
components  of  jobs  and  the  relationship  of  the  job  to  the  organization.  These  analytical 
processes  produce  outputs,  inputs,  and  throughputs  that  are  used  to  describe  these  relationships. 
The  primary,  first-level  product  of  these  processes  is  a  "descriptive”  job  description  produced 
from  job  analysis  data. 

The  job  description  plays  a  critical  role  in  detailing  employer-employee  relationships.  It  defines 
the  mission;  it  provides  a  specification  of  the  expectations  the  employer  has  of  the  employee 
relative  to  the  services  rendered  or  products  produced;  and  it  specifies  the  essential  job  ‘ 
performances  and  required  knowledge,  skills,  and  abilities  (KSAs). 

Because  the  employer-relationship  is  governed  by  a  dynamic  legal  framework,  analysts  need  to 
be  sensitive  to  developments  on  the  legal  front.  The  legal  framework  that  governs  employer- 
employee  relationships  is  built  on  Constitutional  provisions  (First,  Fifth,  Thirteenth,  and 
Fourteenth  Amendments;  Civil  Rights  Acts  of  1866,  1870,  and  1871);  Federal  Statutes  (e.g.. 
National  Labor  Relations  Act,  Civil  Rights  Act  of  1964,  Equal  Employment  Opportunity  Act, 
and  Americans  with  Disabilities  Act);  Executive  Orders  such  as  No.  11141  (1964),  No.  11246 
(1965),  and  No.  11478  (1969);  Agency  Rules  and  Regulations  such  as  the  Uniform  Guidelines 
of  Employee  Selection  (29  CFR  1607)  and  regulations  to  implement  the  Equal  Employment 
Provisions  of  the  Americans  with  Disabilities  Act  (PL  101  336,  29  CFR  1630);  as  well  as  Case 
Law.  Some  of  the  legal  cases  which  support  the  need  for  responsible  job  analysis  and  job 
descriptions  are  listed  in  Figure  12. 

The  military’s  CODAP  system  is  a  job-oriented,  inventory-based  system  that  has  shared  the 
popularity  of  the  inventory  concept  with  the  other  inventory-based  analysis  processes.  The 
utilization  of  CODAP’s  multiple-inventory  philosophy  is  very  useful  when  the  "whole  job"  needs 
to  be  described.  The  atCODAP  is  a  separate  PC-based  system  developed  to  meet  the  needs  of 
smaller  organizations  which  cannot  support  mainframe  CODAP.  The  atCODAP  system  has  a 
companion  product  called  atSURVEY  to  collect  data  on  background;  task  performance;  essential 
knowledge,  skills,  and  abilities;  essential  physical  requirements;  and  essential  job  functions. 

A  pilot-test  of  the  atSURVEY  system  was  conducted  by  this  discussant  to  assess  its  potential  for 
civilian  applications.  The  test  was  highly  successful  and  produced  job  descriptions  that  address 
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Dexler  v.  Tisch  660  F.Supp  1418  (D.  Conn  1987) 

Donaldson  v.  Pillsbury  Company,  554  f.2d 
825  (8th  Cir  1977) 

Guardians  Asso.  of  New  York  City  Police 
Dept.,  Inc.  V.  Civil  Service  Com.  f.2d  79,23 
FEP.  Cas  (BNA)  909,  23  FEP  Cas  Dec. 

(CCH)  P  31154  (2nd  Cir.  1980) 

Hall  V.  U.S.  Postal  Service,  857  f.2d  1073  (6th 
Cir  1988) 

Patterson  v.  American  Tobacco  Company, 

586  f.2d  300  (4th  Cir,  1978) 

Watkins  v.  Scott  Paper,  12  FEP  1991  (1976) 

Weahkee  v.  Perry  587  f.2d  1256  (D.C.  Div. 

1978 


Figure  12.  Job  analysis  and  job  description  case  law. 

the  concerns  expressed  by  America’s  legal  framework.  The  pilot-test  also  indicated  that  the 
following  refinements  to  the  automated  self-report  development  process  are  needed  in  order  to 
make  it  more  useful  to  non-military  users: 

1.  r  values  printed  for  each  inventory; 

2.  V  (validity)  values  printed  for  each  inventory; 

3.  hard  copy  of  the  automated  self-report  survey; 

4.  a  histogram  of  inventory  items,  to  illustrate  overlap  between  job  descriptions; 

5.  a  non-military-user-friendly  menu  system; 

6.  a  non-military-user-friendly  operations  manual;  and 

7.  a  job  description  generator. 
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The  pilot-test  indicated  too  that  the  atSURVEY  system  is  extremely  versatile  and  compatible 
with  all  of  the  computer  platforms  listed  in  Figure  13. 


IBM  PS-2  Models  30,  50,  and  80 
Toshiba  1100  and  1200 
Compaq  386 

Notebook  486-DX  2  50  MHz 
Zenith  286  Portable 
Hyundi  286  Turbo 
ASC  386  SX 
Dell  486  SX  notebook 
Dell  386  DX 

Columbus  Microsystem  486  DX-33 
Next 

IBM  XT  and  AT 


Figure  13.  Computer  platforms  tested. 


[Note:  Additional  comments  and  viewgraphs  from  Dr.  Sage’s  presentation  may 
be  found  in  the  1993  Proceedings  of  the  Eighth  International  Occupational 
Analysts  Workshop  (pp.  85-115).  San  Antonio,  TX:  USAF  Occupational 
Measurement  Squadron.] 
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